You need it to obtain stronger weapons to take down the game's toughest monsters. While in the forge, select the desired sword, axe and what not, then click the Forge button.
Keep in mind you'll need a weapon's blueprint to create it so keep an eye out for blueprints within the game world. What's more, you can upgrade weapons and pendants with Pygite and Cores.
Also, You can forge items anywhere in the game. But if you find a proper forge, creating and upgrading items will require fewer resources and consume less Pygite.