I've always concidered them to be pretty much the same thing, but in this area, 'swap meet' is used for once a year type ones, and 'flea market' is for the weekly or monthly ones.
A car swap meet is a good place to look; I got a big bag full of G1's with a wooden barn for the barbie sized horse at one last year for $25.00.
If you don't find any at the first one you go to, don't give up, getting finds at yard sales, thrift shops, and swap meets/flea markets takes time and a lot of patience.