PET imagery is a valuable oncology tool for characterizing lesions and assessing lesion response to therapy. These assessments require accurate delineation of the lesion. This is a challenging task for clinicians due to small tumor sizes, blurred boundaries from the large point-spread-function and respiratory motion, inhomogeneous uptake, and nearby high uptake regions. These aspects have led to great variability in lesion assessment amongst clinicians. In this paper, we describe a segmentation algorithm for PET lesions which yields objective segmentations without operator variability. The technique is based on the mean shift algorithm, applied in a spherical coordinate frame to yield a directional assessment of foreground and background and a varying background model. We analyze the algorithm using clinically relevant hybrid digital phantoms and illustrate its effectiveness relative to other techniques.