Unfortunately that was the case with the building as reported in the article by Milt Haitema below, which ran in the Lorain Journal on November 5, 1965.
The landmark was located on the northwest corner of State Route 113 (which in 1965 was Lowell Street) and Murray Ridge Road, and was believed to have been as stop on the Underground Railroad.
The article includes a quote by Colonel Raymond Vietzen, who ran the Indian Ridge Museum on Murray Ridge Road from 1930 to 1995.