I had some trouble with this as well. The trick is to attack them with their opposing elements. For fire, use ice. For water, use thunder, use earth, use wind. But you have to be quick! Each crystal tends to attack all characters at once, and the Earth Crystal is meanest (for me). Its "Earthquake" ability killed at least two of my characters each time it was used.
As far as what jobs the party has...I used a Blue Mage with White Wind, the Aero series magic, and Aqua Bubbles (Fire Breath works as well), a Mystic Knight for the Enchating Sword stuff, a White Mage (obvious choice when in a Boss Battle IMO), and a Monk (gotta love Chakra- greedy that it only heals him/herself, but still effective.
There may be a better choice of job assignments, though. I was able to defeat the Crystals with this lineup.