This is an easy problem to solve. First, you can visit Foursquare.com, log in to your account and then click on the Settings tab in the top-right corner of your Web browser. On the following page, you should see a section labeled Your Privacy Profile. In that section, you can uncheck the box next to "Participate in Foursquare mayorships." This will disable your ability to gain mayor titles at Foursquare venues, and it also deletes any mayorships you may have already earned.
Secondly, you can delete individual mayorships by visiting your Foursquare "Me" profile page, locating your list of current mayorships and then clicking the "X"-mark to the right of the mayor listing. You can delete as many or as few of your Foursquare mayorships as you wish in this fashion, and other Foursquare users will no longer be able to tell you visit those specific venues often.
Hide Your Foursquare Contact Information
On the same Foursquare Settings page, you can hide the e-mail address and phone number associated with your account from friends, as well as your Facebook and/or Twitter profiles from all Foursquare users, assuming you've connected them.
Under the "Let my friends see my&" section, ensure that the boxes next to "Phone number" and "E-Mail address" are unchecked. If they are checked, all of your Foursquare friends can access the information, but it will still be publicly blocked.
In the next section, labeled "Let everyone see the links to," you should uncheck the box next to my "Twitter profile and Facebook profiles." This will block other Foursquare users from seeing your connected Facebook or Twitter pages.
Remove Your Name from Venues' "Who's Here" Lists
After you check in to a Foursquare venue, the service includes your name by default on that venue's "Who's Here" list, until you check in somewhere else. That means that anyone who views the venue via mobile app or checks in along with you can see that you're there or were recently--not only your Foursquare friends, but all Foursquare users.
If you'd rather not let any ol' Foursquare user see when you're checked in to a specific venue, again visit the Foursquare Settings page and under Your Privacy Profile uncheck the box next to "Show me in the 'Who's here' list in the mobile app'.
If you connected your Facebook and/or Twitter account to Foursquare, you may also want to uncheck the "Include me in my friends' status updates to Twitter and Facebook when I'm in checked into the same place as they are' option.
Check-in While Leaving a Venue to Avoid Unwanted Visitors