In the old days, there was a special ghillie whose job it was to carry the master across streams. These ghillies were called, ghillie-weetfit. This eventually evolved into a derogatory term for the attendants of a Highland chief.
You can real more about ghillies on Wikipedia's gillie page.
My search was inspired by a BBC article about flyfishing which includes a photo with the caption, "Ghillies have been asked not to keep anything they catch to conserve stocks."
Which made me think, "Ghillie, what's a ghillie?"