In Snow Leopard, Safari defaults to running Java in 64-bit mode. To enable the 32-bit Java required by the Tooth Atlas:

1) Quit Safari

2) In the Finder, go to the Applications folder and Get Info on the Safari icon (right-click OR hold the CMD/Apple key and press I)

3) Mark the checkbox (under the label colors) for 'run in 32 bit mode'

4) Re-launch the Tooth Atlas

===

When you increase java's memory allotment, make sure to do so for both 32-bit and 64-bit modes.