About 14 months after Charlie was born Colonel and his first wife Dorothy had a son and named him after his father. On April 19, 1937 the bastardy suit filed by Kathleen Manson in 1936 was settled in Kathleen's favor. The Colonel was ordered to pay a modest amount child support for Charlie. In 1940 another son was born to Dorothy and Colonel.
April 17, 1941 Dorothy Scott sued Colonel for divorce and won. His drinking problems, abusiveness and non support issues were brought up in court. Dorothy won custody of both boys.
Colonel Scott dies in 1954 of cirrhosis of the liver at age 44. He was survived by his second wife Sylvia and his two sons.