Some other (minor) Purbeck options from Mendip! Registry:
Redend Point Caves (SZ 0380 8280)- Three, on joints in ferruginous sandstone. First record by BEC-member John Noble: Nobel (1981) Sea cave
Parson's Barn Cave (SZ 0530 8220) - One of many, mainly anonymous, sea-caves in Chalk cliffs in the area. Sea cave
Saint Aldhelm's Head Cave (SY 9580 7679) - NGR estimated. V. difficult to locate in NW cliff of headland. Choked, phreatic cave: the only karst cave so far known in Purbeck. Karst cave
Kimmeridge Oil-Shale Adits (SY 9201 7770) - Fast-disappearing remains of oil-shale mines in sea-cliff. Artificial
Pondfield Cove Cave (SY 8715 7956)- Awkward access (short swim or boat). In Portland Formation limestone: may be karst but sea-cave assumed. Sea cave
A bit far, but could be combined if investigating swallets around Tisbury is Chilmark Quarry at ST 974 312 (https://bec-cave.org.uk/index.php?option=com_content&view=article&id=975:belfry-bulletin-no-327-january-1975&catid=68&limitstart=3&Itemid=558&lang=en)