Sim Whittier Page
Welcome to Joe's Pond Vermont
Sim's Island on Joe's Pond
by E. Jane Brown
(Originally published in The Caledonian Record, Sept. 23, 1994)
In the mid 1700's, the area surrounding what we now call Joe's Pond was visited regularly in the summer by Indian tribes from Canada. According to William Parker Neal in his booklet, "Indian Joe, The Friendly Indian Guide", published in 1976 by the United Methodist Church of West Danville, Vermont, one of them, who called himself "Joe Indian," was friendly to the early settlers, warning them when there was to be an attack. Once the Indians found out what Joe was doing, he had to flee for his life. His pursuers believed there were evil spirits in the pond so did not dare venture onto its waters.
It is believed Joe Indian and his wife, Molly, may have found sanctuary on an island in the northwest corner of the pond that is named in his honor. The island is still called "Injun Joe's Island" by some, but it is also known locally as "Sim's Island."
Sometime before the turn of the century, Rufus Simeon "Sim" Whittier returned from sailing the seas and fighting Indians out west, and like Joe Indian, sought haven at Joe's Pond. The area had become settled and at the outlet of the pond, in West Danville, a dam had been built. This raised the level of the water so Indian Joe's Island was little more than a mound of earth and some ledges sticking out of the water. Sim Whittier chose to reclaim the island for his home.
Aaron Bolton, 88, of Cabot, said his father told how Sim brought rocks and earth from the mainland, filling his wooden boat until it would barely stay afloat. He rowed load after load out to the ledges and built a stone wall to form the perimeter of what would be his island. He then filled in the center with rocks and earth until it was large enough and high enough to accommodate a house. He then built himself a small two-story home, made snug with plastered walls, added a chicken coop on the north end of his little island, and named his home "Indian Joe's Wigwam."
Sim lived on his island for about 43 years, until his death in 1937. He was a mild-mannered man according to those who knew him, and he was always willing to help his neighbors any way he could.
"He was not a very big man, and I remember he had kind of a high voice," Bolton recalled. "He was always working on somebody's boat - you'd see the boats pulled up onto his island and he'd be fixing them up, calking and painting them, and he could splice a rope so smooth you'd hardly know when the splice went through the sheave."
Rufus Sim Whittier was born November 7, 1849 in Lowell, Mass. His mother, Sarah Heath Whittier, was born in Cabot, Vermont, but little is known about his father, Sam Whittier. He may have died when Sim was young, which could account for Mrs. Whittier bringing her children back to Vermont.
At the age of 20, Sim joined the U. S. Navy, serving as a messenger. He visited ports in Cuba, Haiti, Bermuda, Puerto Rico, Europe and the Pacific. Sim left the Navy in 1871 and returned to St. Johnsbury, Vermont, where he found his mother very ill and his 11-year-old sister, Nellie, caring for her. He found work with the railroad as a trackwalker at the West Danville car house and stayed in the area until his mother died in the winter of 1872. Sim then took Nellie to Boston, probably to live with relatives, and he worked for a furniture manufacturer there for a short time.
In 1873, Sim joined the military again, this time serving in Co. H., 17th U. S. Infantry in North Dakota Territory with General Custer. Sim was with a small group of soldiers ordered to remain at their encampment while General Custer and 190 of his men scouted for bands of Indians. This was part of a planned attack under General Alfred Terry, but history tells us when Custer found the Indians, his small regiment was outnumbered and destroyed. Sim told how, when the regiment did not return as expected, he and the others set out to find their comrades. They lost 71 men in skirmishes with the Indians, and when they arrived at the site of the Battle of Little Big Horn, they found the grizzly evidence of the massacre that was "Custer's Last Stand."
Two years later, Sim gave up fighting Indians and started back east by covered wagon. There is no information of his whereabouts for the next dozen or so years. It may be that he encountered some sort of mischief along the way, because Aaron Bolton recounts that Sim once told his father that he was AWOL from the French Navy. Sim said he escaped from the French by swimming ashore one dark night as the ship sailed close to land. It wasn't until after Sim had lived here in Vermont many years that the elder Bolton related the story, but it might explain at least some of those missing years and why Sim chose to live alone on a remote island in northern Vermont.
Sim was very much a local hero when he returned to Vermont with his vivid, first-hand descriptions of Indian fighting, and eventually he became Vermont Commander of the National Indian War Veterans.
Sim was an expert fisherman and shared his catches with his neighbors. He worked at odd jobs, carpentry or plastering, painting or wallpapering. He would row his boat to the east shore and walk the railroad tracks to get to town. At either Hasting's or Milligan's store in West Danville, he'd purchase a bottle or two of "Beef, Iron and Wine Extract," and before starting home he'd take a long drink of the stuff. Ralph Hastings once told how Rose Smith, who lived on the farm by North Shore Road, would sometimes remark, "Sim's pretty full tonight," as she watched him stumble along the tracks to where his boat was tied up. This was during prohibition and Sim apparently had a source to get liquor, because folks say he always kept a bottle or two anchored just below the surface of the water somewhere between his island and Priest's Island. When he went into town, though, he had to improvise.
Sim was not one to be taken advantage of. When the Town of Cabot passed a special regulation imposing a tax for the maintenance of roads, Sim refused to pay, saying he had no need for and did not use the roads, therefore should not have to pay the tax.
Most of the years he was on the island, he stayed year 'round. When his health began to fail, he moved to the "Republican Block" in St. Johnsbury during the winters and returned each spring to his island. Sim died at St. Johnsbury Hospital on July 8, 1937, at the age of 87, of "chronic myocarditis with cancer of the intestines." He is buried next to his mother in the Cabot Plains Cemetery.
There were a few brief owners of the island after Sim's death, but no one lived there. In the late 1940's, Phil Rogers and Arthur Bartlett of Cabot moved Sim's little house off the island, taking it on the ice to a lot Bartlett owned on the west shore. Bartlett eventually sold the property to Muriel Blanchard of Barre, Vermont. (Note: Ms. Blanchard eventually sold to Marcel Rouleau, of Barre, and it is now, in 2007, owned by Marlynn Rouleau, of Barre.)
After the house was moved, the island was abandoned until 1955 when William H. "Bud" Pearl and his wife, Lucia, of St. Johnsbury, bought it for $150. They turned the single remaining small building into a cottage and spent many summers there while their four children were growing up. The Pearl family still goes to the island, now with a third generation of children. It is much the same as it always was except some of the big spruce trees are gone and the island has been diminished by high water and erosion.
Sim's sister, Nellie, was great-grandmother to Alice Calkins of Danville. Calkins said she never knew Sim, but she has heard stories about him.
Although "Simmy" lived long enough to see the area change over the years - more cottages were built at the pond and motorcars and motorboats came into use - he would probably be surprised that today nearby Priest's Island has electricity and telephone service. He might also be surprised and perhaps saddened that trains no longer run regularly on the tracks he walked, and there are few wooden boats left to be caulked. The broad, fertile fields and open pastures sloping to his beloved pond have grown up to forest again, and the few open remaining are mostly fallow land or manicured lawns, and there are few farmers left to need ropes spliced or hay wagons repaired.
There are lots more people around Joe's Pond these days than when Simmy lived here, and gradually more of them are deciding to stay year 'round. That might please him - or maybe not.