You might have flea beetles. Here's what you do:
If you see that you have a whole bunch of holes in your broccoli leaves and want need to stop the damage then spray Bt (Bacillus thuringiensis, which is sold as Dipel, Thuricide and probably other names). It is a bacterium that infects only certain insect larvae and nothing else, so it is quite safe in your garden, won't hurt honeybees or any other adult forms or non-insects. - Birdy