You can enter Abkhazia from both Georgia and Russia. However, if entering from Russia you would need a double entry Russian visa, as you can't continue on to Georgia (Georgia still consider Abkhazia part of Georgia, so from their perspective you would have entered Georgia illegally by coming from Russia). So most non-Russian visitors arrive from Georgia, across one of the weirdest borders in the world.
You are not getting stamped out on the Georgian side, but you still need to register at police post and show your Abkhazian visa approval. You then walk 1.5 km - or pay for a ride in a horse cart - through no-man's land on a crappy road, which is only used by cows and Abkhazian border shoppers, who have been to Georgia to pick up a new television or aircon. Right before the Abkhazian border post you cross an old bridge, which is guarded by a few bored looking Georgian soldiers. At the first Abkhazian post the officer will call the Foreign ministry to check your visa. When confirmed, you walk into a fenced off passage, which leads to the final check point, where your whereabouts will be questioned. Finally you're through to the Abkhazian side. There are nothing else on both sides beside waiting taxi drivers and the occasional marshrutka. No money changers, no shops, no nothing.