Bobbleheads provide either a permanent +1 bonus to a SPECIAL stat or a permanent perk of some kind. This is one of the very few ways you can upgrade your SPECIAL points so that you don't need to waste a perk point on raising a stat just to access a specific perk you want.
By the way, you can construct a bobblehead stand at a settlement and then transfer over any bobbleheads you have for a nifty display. You will earn an achievement for collecting 10 as well as 20 of the bobbleheads.
Here are all the locations for the bobbleheads in Fallout 4.
Location: Mass Fusion Building - The lobby desk should has a metal wall sculpture floating above it. This bobblehead hides on top of the sculpture.
Effect: Increase Strength by 1.
Quote: "It's essential to give your arguments impact.
Location: Museum of Freedom - This will likely be the first bobblehead you find since it's right where you meet one of the first companions in the game. This bobblehead is resting on desk right near where you meet Preston Garvey for the first time.
Effect: Increase Perception by 1.
Quote: "Only through observation will you perceive weakness."
Location: Poseidon Energy - First find the central metal catwalk hut in the plant. There is metal desk with this bobblehead resting on top of it. You'll need to kill a lot of enemies and turrets as well as a Lvl 39 boss.
Effect: Increase Perception by 1.
Quote: "Always be ready to take one for the team."
Location: Parsons State Insane Asylum - Once you have access to Jack Cabot's office, this bobblehead rests on his desk in the asylum. If you have the chance to head to Bunker Hill, you may receive the questline to speak with Jack Cabot.
Effect: Increase Charisma by 1.
Location: Wreck of the FMS Northern Star - This bobbleheads rests in an extremely precarious spot, right on a wooden platform along the edge of the ship.
Effect: Increase Agility by 1.
Quote: "Never be afraid to dodge the sensitive issues."
Location: Boston Public Library - This Super Mutant-filled location can be a bit tough to navigate, but there is a mechanical computer room near the northwest corner of the Library. The bobblehead rests on a computer bank there.
Effect: Increase Intelligence by 1.
Quote: "It takes the smartest individuals to realize there's always more to learn."
Location: Spectacle Island - There's a green boat along the southern shore of Spectacle Island. This bobblehead is a lock next to a streamer trunk.
Effect: Increase Luck by 1.
Location: Longneck Lukowski's Cannery - In the northwest upper area of the main cannery room, there's a metal catwalk hut that has this bobblehead inside.
Effect: Prices are permanently 5% better.
Location: Vault 95 - This bobblehead isn't tough to find within the Vault as it rests on top of a radio within the north-most room in the living quarters area.
Effect: Gain +25% critical damage with heavy weapons.
Location: Fort Hagen - Yet another Bobblehead you'll come across through the main storyline, this Bobblehead can be found in the kitchen area near the end of the dungeon in between two refrigerators.
Effect: Gain +25% critical damage with energy weapons.
Quote: "Arrive at peaceful resolutions by using superior firepower."
Location: Saugus Ironworks - After clearing the factory, there will be another section called Saugus Blast Furnace area that has a boss. This bobblehead is on a platform near when the boss speaks with you.
Effect: Gain +15% damage with explosives.
Quote: "The best way to solve a problem is to make it go away."
Location: Pickman Gallery - If you clear the area, you'll find Pickman waiting for you in the basement of the area. This bobblehead is in the area where you find him, so it's easy to miss. The Pickman Gallery questline can be found by talking to Hancock in Goodneighbor.
Effect: Lockping is permanently easier. (This likely means that the sweetspot for a successful lockpick is larger than before.)
Quote: "Always strive for the unobtainable."
Location: Trinity Tower - This is yet another Super Mutant-area and at the top you'll save two characters. The cage they're in has the Melee bobblehead inside.
Effect: Gain +25% critical damage with melee weapons.
Quote: "It's important to do business up close and personal."
Location: Vault 81 - This bobblehead is in the Secret Vault area of the Vault (a part of the Hole in the Wall quest). The robot can grant you access to the room's door.
Effect: Stimpaks permanently heal 10% more damage. (You may want to consider not taking all four ranks in the Medic perk, which has Stimpaks heal 100% of damage, if you grab this.)
Quote: "The smart man knows a bandage only hides his wounds."
Location: Corvega Bobblehead Plant - If you can reach the top of the plant, you will be able to locate a spherical storage tank with this bobblehead located at the end of the walkway.
Effect: Fusion cores last 10% longer.
Quote: "Why go down with the ship when you can try to fix it?"
Location: Vault 75 via Maiden Middle School - Once you have access to Vault 75 through the Maiden Middle School, you'll eventually come across a room where you can see the combat course and enemies with Admin access card keys. This bobblehead is in this room.
Effect: Get one extra guess when hacking terminals.
Quote: "Always be prepared to explain the hows and the whys."
Location: Gunners Plaza - This bobblehead hides in the broadcasting room, which you can access by having Master-level lockpicking (3 ranks in Lockpicking perk) or by finding the key by killing one of the enemies in the basement.
Effect: +25% critical damage with ballistic weapons.
Quote: "Because it's easer to have courage from a safe distance away."
Location: Dunwich Borers - Delving deep underground, you will find a foggy area with a post terminal. The bobblehead is right next to it.
Effect: You are 10% harder to detect.
Location: Vault 114 via Park Street Station - You'll find this Bobblehead along the main campaign. In the vault, you'll meet one of the game's possible companions. In this Overseer room, you'll find this bobblehead.
Effect: All vendors have 100 more caps for bartering.
Location: Atom Cats Garage - The main warehouse of the garage has a rusty car that has this bobblehead on top of it.
Effect: Gain +25% critical damage with unarmed attacks.