In the western part of town, you can find a hidden Zinc on the ground. Professor Juniper's lab is also on the northern part of town. Stop by there and she'll check to see how many Pokemon you've seen for your Pokedex. If you've seen all of the Pokemon in the new Unova Dex (#001-297; this excludes Victini, Keldeo, Meloetta, and Genesect), she'll reward you with the Permit, which can be used to travel from the Mistralton City airport to the Nature Preserve, where you can find a Shiny Haxorus. It's definitely a goal you should make, because a Shiny Haxorus is pretty cool.
If you complete your Unova Dex by catching #001-297, you'll receive the very rare Round Charm Key Item, which increases the rate your Pokemon produce eggs as long as you have it in your inventory. That is obviously very useful to have if you do a lot of breeding and makes for a nice reward for a hard task.
Additionally, Cedric Juniper will check your entire National Pokedex for Pokemon caught (as opposed to seen). If you've completed your National Dex, he'll reward you with the Shining Charm, which is similar to the Round Charm in the fact that it's a Key Item that has a constant effect, but the effect of the Shining Charm increases the odds of encountering a Shiny Pokemon substantially, reducing the odds of just 1/3 of what they were before (1 in 8192 down to 1 in 2730). That's substantial, so it's very worth completing your Pokedex in Black 2 and White 2, since you receive something far more useful than just a diploma.
Regardless of your Pokedex count, though, Cedric Juniper will give you the Super Rod after talking to him for the first time, which you can use to fish for more Pokemon. You can also fish right towards the rippling water, casting your rod into it, to potentially fish up much rarer Pokemon.
The house in the middle of town is where the old hero from Black and White once lived; you'll run into his or her mom, who reminisces about N. If you performed a Memory Link, she'll talk more about your character from the original Black or White version. Otherwise, she'll be more vague. She will heal up your Pokemon whenever you talk to her, too. Upstairs, be sure to pick up the King's Rock, which has a chance of causing the foe to flinch if you make a direct attack, plus it can be used in trades to evolve certain Pokemon, such as Poliwhirl and Slowpoke.
The house in the southwest part of town is Bianca's family's house, although they'll do nothing more than just talk about her being Juniper's assistant.
The southeastern house is Cheren's family's house. Talk to his dad and he'll give you some Berry Juice. Well, that's just great.
Anyway, now that you have your Super Rod, head back north onto Route 1. Make sure you have a Pokemon that can Surf, too!