Save up you baits for a future fishing party. Fishing in group increases the chance to catch the rare fish. Each zone has 12 rare fish and each starter zone has one.
You need to catch all rare fish to get the Master Angler achievement. Catching all 12 from each zone gives you achievements also. The achievements unlock dye colors for your armor.