Peter Swailes Jr, 56, admitted the offences after his vulnerable victim was rescued from a rotting 6ft shed with no lighting, heating and a soiled duvet.
He was made to live in a horse box, a disused caravan and more recently in the shed in Carlisle.
Swailes Jnr admitted the offences on the basis he was unaware of the victim's living conditions.
He was sentenced at Carlisle Crown Court to a nine-month jail term, suspended for 18 months.
Swailes Snr, who died last year while awaiting trial, denied the offence.
The charges followed a three-year investigation by the Gangmasters and Labour Abuse Authority (GLAA).
Prosecutor Barbara Webster said: "When the victim was around 18 years of age, he was approached by Mr Swailes Senior, and invited to work with him doing various jobs.
"He lived on the same site as Mr Swailes Senior.
"He had various accommodations, including a horse box, an old caravan, and in the last five years or so, and during this offending, the shed that we have heard so much about.
"The court will be aware that he lived in the shed next to the chalet of Mr Swailes, with no heating, no lighting and no flooring.
"In stark contrast, the family dog lived in much more comfortable surroundings."
Miss Webster said the victim had a low IQ and "very little understanding of the world around him".
She went on: "He was found by the police living in a rotten shed, with water pouring through it, with a make-shift bed, and congealed vomit in the corner.
"Not the way that anyone would choose freely to live and not where he would be if he could have found himself better living accommodation.
"He had few possessions to show for his 40 years' hard work. He only had a wash bag, three second-hand coats, a few stained duvets, and CDs."
He lived with the Swailes family for many years after being in care as a child, with Swailes Snr telling him he was his "boss".
When he was not being made to work, Swailes Snr would order him to do chores such as painting the kitchen and cutting the grass.
In a nearby similar-sized shed to the victim's "accommodation", there was a fitted carpet, a light, a gas heater and the family dog.
Miss Webster added: "Peter Swailes Senior had a far better standard of living, an elaborate, carpeted home with expensive personal belongings. A palace by contrast to where (the victim) lived."
When officers attended the traveller site in October 2018, the victim told them he only received £10 a day for his work duties.
He received specialist help after he was rescued and now lives in supported accommodation outside Cumbria.
Martin Plimmer, of the GLAA, said he had been sleeping in conditions "that no human being should live in".
He added: "This has been a really harrowing investigation. In all my years in law enforcement, I have never known a modern slavery case where the exploitation has taken place over such a long period of time."
A man was held captive in a small shed for 40 years
Free: Peter Swailes Jnr leaves Carlisle Crown Court