Just outside of Phetchaburi you can find Khao Luang cave. Its a fairly remarkable place with huge vaulted caverns and large numbers of Buddha statues which have been added over the generations. Apparently this place received its first big boost during the reign of King Rama IV, who spent a lot of time in the area. Since then it has developed into a popular destination for Thai tourists.
Getting to the caves can be a little awkward. I'm sure most songteaws in town would know where the cave is as its only 10 minutes outside of town. I don't know how much that should cost however. We had a motorcycle so we were able to run out there ourselves. It took a little bit of finding but we did track it down.
Along the way we passed through a whole bunch of monkeys with more of them hanging around the car park at the cave. The monkeys out there seem to be a little less aggressive than those in town. I would however suggest that walking out there would be a bad idea. The distance is certainly walkable if you have some time. However, we saw a few people delivering food for the monkeys and they certainly came running at the sound of people moving around. If you show up on foot with no food you would disappoint the monkeys and that would make them sad. Sad, and hungry...
Once you arrive there appear to be two paths from the car park that lead to two different cave entrances. One is for the public and leads directly to the main cave entrance. The other appears to go via monk's accommodation and, we believe, leads to the smaller entrance to the cave. The one you want, the entrance for the public, is at the back of the car park.
If you're in the area definitely check out Khao Luang Cave. Its a very beautiful, serene place with a long history.