For a figure who would one day garner the title “Witch of Wall Street,” Hetty was a particularly lovely young woman. She was tall and full-figured, with large blue eyes, a long, straight nose, prominent chin, and generous brown hair. Her favorite place was still with her father in the counting house or on the docks, and at times she used waterfront language that shocked genteel souls. Aunt Sylvia, in particular, fretted over Hetty’s lack of preoccupation with feminine things, and worried that, even by the modest standards of Quaker dress, she stood out as unfashionable.
When she was about twenty, perhaps at Sylvia’s urging, Hetty spent a month in New York as the guest of Henry Grinnell, her mother’s cousin. New Bedford’s prosperity was well known throughout the country, and the wealthy of New Bedford found access into the upper circles of New York business and society. Born in 1799, Grinnell had left New Bedford for New York as a young man, joined a mercantile business started by his brother, and established himself as one of the city’s most prominent merchants. A worldly man and an adventurer at heart, Grinnell financed several Arctic expeditions and served as the first president of the American Geographical Society.
Theirs was a lively house of six children (three more had died young). Daughters Sarah and Sylvia instructed Hetty on New York society and introduced her to their friends. Despite Aunt Sylvias misgivings, Hetty had picked up a thing or two at Mrs. Lowell’s school and could behave like a lady when she wanted. She attended balls, luncheons, parties, and concerts, and she turned the heads of young men, not just because of the money she stood to inherit, but because of her beauty. By all appearances she enjoyed herself enormously. In later years she remembered in particular one glittering affair—a dinner at Saratoga Lake, at which Martin Van Buren, the former president, and his son honored visiting English royalty, including Lord Althorp, who later became duke of Northumberland.
In 1860, Hetty wore a low-cut white ball gown with a pink sash and lace trim to a ball held at the New York Academy of Music. She wore pink slippers, long, white, kid gloves, and gold earrings. She carried an ostrich feather fan. The reason for the ball was a visit by the Prince of Wales, the future King Edward VII of England, who was in the midst of an extended tour of North America. At the ball, Hetty had herself introduced to the prince as “the Princess of Whales.” The prince appreciated the joke. He laughed and told her, “I’ve heard that all of Neptune’s daughters are beautiful. You are proof of that.” They danced twice.
With her social connections, her looks, and her family wealth, Hetty, had she chosen to do so, could have shed the straitlaced provincialism of New Bedford and entered seamlessly into a life of ease in New York society, of summers at Newport and winters on Fifth Avenue. After a short stretch as a debutante she might have married a steel prince or a railroad king and spent her life raising her children with the help of French nannies. She might have organized fashionable balls to benefit the poor, held choice seats at the opera, and ridden elegant carriages around New York and, when her beauty faded, taken her place among Edith Wharton’s gallery of proper New York dowagers.
But something in Hetty Robinson drew her home, back to her dour family and the seat of her family’s money. Her father sent her $1,200 with instructions to properly outfit herself with dresses and gowns for the social season. Hetty spent only $200 of the money. She put the rest into the bank upon her return to New Bedford.
Between her schooling on Cape Cod and in Boston, Hetty had lived periodically with her parents in a house they had purchased on Second Street. But Hetty always maintained a bedroom in her aunt Sylvia’s home, and stayed there much of the time. As Abby and Edward’s marriage froze over, Abby herself spent much of her time at her sister’s home, seeking refuge from her cold, ill-tempered husband. But even when Abby and Hetty shared a house, Abby was little more than an adjunct parent to her own daughter. And when Abby died, intestate, on February 21, 1860, the effect on Hetty seems to have been something akin to losing a close but not particularly vital relation—a spinster aunt, perhaps. Edward took over virtually all of her $128,000 fortune, with the exception of a house valued at $8,000 that went to Hetty. Abby’s relatively modest holdings at the time of her death, given that she was one of two natural heirs to a whaling fortune, only underscored the extent to which Edward had dominated his wife. From what little is known of Abby’s personality, she handed down almost nothing in the way of personality traits to Hetty; they all came from her father.
It is the actual spinster aunt, Sylvia, Abby’s sister, who occupies a far greater position of importance in this story. Sylvia suffered for decades from a spinal disorder that had apparently started when she was very young. Born into wealth, she never had the health or energy to derive much enjoyment from it. Her condition precluded marriage and travel, and so she found refuge in books. She sat for hours as nurses read to her, carrying her to distant shores with the works of popular authors of the day such as Frederika Bremer and Bayard Taylor. Nurses kept her apprised of world events by reading to her from her favorite newspaper, the Boston Journal. She tried her hand at poetry. A poem handwritten by Sylvia survives in the collection of the New Bedford Free Public Library, in small, delicate cursive:
In your Album I descry a page
On which no pen has left its trace
I will endeavor to portray
A wish that may not be erased
May much happiness attend you
And the love of God may you implore
May this blessing rest upon you
And His name may you adore
The poem is dated December 4, 1845, when Sylvia Ann was thirty-nine years old. The Esther of the title remains a mystery. The poem is signed “Sylvia Ann Howland”—a signature that, on different documents and a couple of decades later, would lie at the center of a raging controversy.
As she aged, and especially after the death of her sister in 1860, Sylvia lived a life of Gothic loneliness. She shuttled back and forth between her large house on Eighth Street in New Bedford and Round Hill farm, the family estate. Her world consisted of a few friends and relatives who called on her at the farm or in town, Thomas Mandell, the minor partner in Isaac Howland Jr. and Company who managed her estate, and the round-the-clock attentions of a few loyal servants and nurses.
She was obsessively needy of these paid friends. When they left her for even a few moments, she became ill at ease and implored them to return They were her companions, her world. Gradually, at her request, a small coterie of them gathered around her, gave up their outside work, and built their own worlds around hers. It was a clean trade of full-time attention in exchange for steady, full-time employment. Pardon Gray, a New Bedford livery stable owner and hack driver, was by 1855 driving exclusively for Aunt Sylvia. Pardon’s most frequent route was the seven-mile journey from New Bedford to Round Hill, which took two hours over country roads. In the summer, Pardon and his charge would leave New Bedford by eight-thirty in the morning to beat the heat. In other seasons they would leave at ten. When they arrived at Round Hill, they would eat at the same table together, along with the nurses. There was little pretension in this household.
Fally Brownell did the cooking and housekeeping. She had been with Sylvia Ann since 1842, and hence had known Hetty as well almost all of the girl’s life. Another vital member of Sylvia’s staff was Electa Montague, who had arrived as a nurse for Abby in 1859, when she was staying with Sylvia. Following Abby’s death in 1860, Electa stayed on to care for Sylvia.
With Sylvia unmarried and childless, Hetty was the sole blood heir to the Howland whaling fortune. As such, she saw Sylvia not just as an aunt, but as the caretaker for a fortune that would one day pass on to her. She began to fear the influence that the coterie of nurses and servants might have on Sylvia. Sylvia was weak and often indecisive. How could Hetty be sure that some servant wouldn’t swindle her out of a chunk of the estate? Her gnawing fear quickly developed into an obsession, until she could barely stand the thought of Sylvia being alone with her staff.
In particular, she resented Fally Brownell, the cook and housekeeper. Sylvia trusted Fally implicitly, to the point that she gave Fally the key to a large, hair-covered trunk that Sylvia kept in a closet in her bedroom. The trunk held some clothes, jewels, and other belongings, but it also held money and financial papers. Sylvia for years had kept the key to herself, retrieving money or papers from the trunk as she needed them. But since she had grown weaker, about 1859, she had turned the keys over to Fally, making Fally, now in her late sixties, the de facto keeper of the household money. Fally kept the keys locked in a trunk of her own, under strict orders not to give them to anyone, including Hetty, without her employer’s specific instruction.
This intimate show of trust between Sylvia and Fally infuriated Hetty and deepened her paranoia. And that paranoia only intensified in 1861, when Edward Robinson decided to move to New York City, asking his twenty-six-year-old, unmarried daughter to join him there. What devious plans might Fally and the rest of them devise with Hetty more than two hundred miles away?
Robinson’s decision to leave New Bedford coincided with his decision that same year to close up operations of the venerable Isaac Howland Jr. and Company. As always, his business timing was impeccable. He had entered the whaling business in the 1830s, just as it was approaching its zenith, and now he quietly but quickly sold off the ships and other assets when he recognized unmistakable signs that the industry was headed for a decline from which it would not recover. The recent discovery of oil in Pennsylvania, combined with new techniques for refining it, promised a new and seemingly inexhaustible supply of oil that didn’t require costly, dangerous voyages on the far seas. But another factor greatly hastened the downfall of whaling—the Civil War.
In New Bedford, the war had started—as wars invariably do—amid a burst of optimism and euphoria. In April 1861, just days after the Confederate attack on Fort Sumter, former Massachusetts Governor John H. Clifford whipped the citizenry of New Bedford into a patriotic fervor with a speech in front of City Hall, promising “untarnished glory” and “hearty joy and honor” to enlistees. A Ladies’ Soldiers’ Relief Society was quickly established, collecting flannel shirts, blankets, mittens, quilts, preserved fruits, coffee, tea, cocoa, lemons, brandy, woolen socks, undershirts, and dozens of other goods in bulk to support the troops. The state called on New Bedford, based on its population, to provide 2,100 soldiers to the cause. In the end, New Bedford sent 3,200, several hundred of whom never returned.
Whalers fortunate enough to assemble working crews during the war found themselves besieged by Confederate raiders. Confederates destroyed no fewer than twenty-five New Bedford ships, seizing or destroying a half-million dollars’ worth of oil. But the coup de grace to the whaling fleet came not from the Confederate side, but from the Union. In the fall of 1861, the United States Navy commandeered thirty whalers, most of them from New Bedford, filled them with stone, and sank them in the shipping channels of Charleston and Savannah to blockade Southern shipping. The terrible truism that war is good for the winning side’s economy never played out in New Bedford, which—more like a Southern city than a Northern one—had to swallow economic disaster right along with the human tragedies of battle.
Some New Bedford whaling men, attached to the city and to their way of life, would ride the industry down to its bitter end in the early 1900s, sending fewer and fewer ships out to hunt ever more elusive whales, for an oil that the world no longer wanted. But sentimentality was never one of Edward Robinson’s traits. To him, whaling had been nothing more than the best way to make money, and New Bedford the best place to do it. When the time came, he cast them both off with the brisk indifference with which another man discards a worn-out pair of shoes. In New York, he joined the New York shipping firm of William T. Coleman and Company, where he found new success in merchant shipping and real estate. Hetty moved to New York with her father, but she returned frequently to New Bedford to keep an eye on things.
Each time she came, Hetty blew into and through the insular, sequestered, quiet world of Aunt Sylvia like a fresh gale. She stayed with Sylvia on Eighth Street or at Round Hill. Hetty had acquired a home of her own, the house on Second Street where Edward and Abby had lived, after her mother’s death. Her father let her have it as a sort of consolation prize, after assuming control of all of Abby’s money. But Hetty preferred to stay with Sylvia. Whether this preference was born of a fondness for her aunt and a desire to be the dutiful niece, or a self-centered desire to keep tabs on the cash cow, depends on who was telling the story.
Though she had as yet no legal claim on any of Sylvia’s money, she felt no compunction about monitoring Sylvia’s spending habits. However unattractive and at times irrational Hetty’s behavior toward Sylvia was, personal greed did not seem to be among her motivations, at least, not greed in the conventional sense. She did not lust after Sylvia’s money in order to one day shower herself with luxury. Indeed, the miserly habits for which she would become famous later in life point to the opposite extreme, an abject unwillingness to enjoy the indulgences that other wealthy people (or even members of the middle class) took for granted. Her concern with Sylvia’s money grew instead out of her own obsession to protect the family fortune at all costs, and from her belief that the money would not be safe until it was in her hands.
What is clear, though, is that Aunt Sylvia’s staff, and Sylvia herself, increasingly came to dread Hetty’s visits. Physically and emotionally, Aunt Sylvia was no match for her energetic, assertive niece. Sitting in a wheelchair or in her bed, her small body curved by the disfiguring spinal condition, she felt at times helpless in Hetty’s presence. And yet it is clear that Sylvia held her own in this complex relationship. Hetty hectored her aunt about expenses she felt were unnecessary. Particularly galling was Sylvia’s plan to add a new section to her Eighth Street house. Hetty believed that the staff was brainwashing Sylvia into adding extra rooms solely for their own comfort.
“There’s plenty of room in this house!” she cried. “You don’t need to add on just to accommodate some old nurses!”
When Sylvia said she planned to go ahead with the addition regardless, Hetty threw a tantrum, sitting on the floor in front of her aunt, sobbing. When that approach failed, she marched dramatically upstairs to the room where she usually stayed, a spacious bedroom directly over the kitchen. She grabbed her belongings from the closet and bureau and took them by the armload up to the attic. If her aunt was determined to make an addition to the house—if the servants were twisting her withered arms, as she believed they were—Hetty would show by example how little space she herself required.
Heading up the attic steps, she declared, “I’ll never sleep in that old chamber over the kitchen again! I would rather sleep in a cemetery.”
Electa stared evenly at Hetty.
“If you want to sleep in the cemetery, go there.”
“I shan’t go there,” Hetty replied, “until I’m carried.”
With that, Hetty stormed up the steps. She took a mattress from the bedroom and laid it out across her grandfather’s old sea chest and a storage trunk. Hetty’s strike lasted precisely one night. When the house failed to be swayed by her display, she quietly moved her belongings and mattress back to her old room the next morning.
One day, Hetty visited the home of a cousin in New Bedford during a snowstorm. Sylvia sent a carriage to bring her home. Hetty refused the ride as wasteful—she insisted on walking. Another time, Hetty decided to have a party for some New Bedford friends and relatives. Sylvia agreed to host the party, but only on the condition that Hetty would agree not to pinch pennies on the food and decorations. But the two were soon arguing over the number of chickens and the quantity of ice cream required to adequately entertain guests. “That’s too much!” Hetty shouted. Sylvia wanted expensive lace doilies; Hetty put out cheap cotton ones. Instead of hiring a waiter for the occasion, Hetty wanted to borrow a servant girl from another family. Mortified, Sylvia declared that she would never again allow Hetty to entertain in her home.
For her part, Sylvia carped at Hetty over her unbecoming clothes, and her less than astute attention to her appearance. If Hetty paid half the attention to her appearance that she did to financial matters, Sylvia reasoned, her niece would be a charming young lady. As it was, Sylvia complained to her nurses, to Hetty herself, and to Pardon Gray, the driver, who was raising daughters of his own. Once, Sylvia said wistfully to Gray, “I’d give a great deal if Hetty was like other young ladies, like your girls, or a great many others. Her dress plagues me, going down the street looking so. It’s the talk of the town.”
Letters from Electa to Hetty, written in rough grammar and spelling when Hetty was in New York in early 1864, reveal both Sylvia’s concern over Hetty’s hygiene and a genuine love sometimes overlooked in accounts of their relationship: “Dear Hetty, your not[e] cam[e] safe. Very glad to hear of your safe arrival your Aunt is so very glad that you have got such nice rooms[,] to have them warm to[o] we can talk about you almost see you made comfortable it gives us great joy. She thinks of that nice warm quilt you have to[o] it is [a] great pleasure to think of you and to have you dress nice and clean.”
Four months later, Electa wrote another letter, expressing Sylvia’s concern that Hetty had neglected to take with her a cashmere shawl that Sylvia had hoped would spruce up Hetty’s wardrobe:
“Your Aunt gave you that bleak [sic] Cashmere shall [sic] that you wore last fall, expected you to take it, found you had not[;] it is such nice suitable shall for you to wear, she wants you to have it and wear it and look like a lady in your place not keep it she says she never shall wear it. She has not been as quite well since you left nearly the same. I hope to carry her out doors soon. She sends her love to you with all the rest of us and hope you are getting house cleaning setting down your things finely write soon and very often.”
The letter included a postscript that constitutes one of the few hints that this insular, self-absorbed world was even aware of events raging beyond their doors: “O this awful war Oh how many are killed.”
Hetty’s letters, which would emerge years later as evidence in a court battle over Sylvia’s estate, profess love for Sylvia, Electa, and Eliza Brown, another nurse. But they also show the conflicts and bitterness seething through the relationships. There is one letter, undated but written by Hetty presumably from New Bedford to Round Hill. From the content of the letter, it appears that Sylvia has decided to proceed with the addition to the house, and that the carpenters have shown up on Sylvia’s orders, much to Hetty’s consternation. The letter shows that she feels remorse, after a fashion, for her outburst, though it is couched in self-pity. “Dear Monte,” she wrote, using a nickname for Electa Montague, “I am almost crazy I have my old head aches and so discourage. I had tried to be so good—and to be deceived as well as have it done was too bad. I cared more about her not telling me…. give my love to Aunt and tell her the carpenters are here as it will make her happy although it will take me two years at least to get over the shock. Why did you let her tell me so sudden you would scold me if I treated her so…. I never wanted her to give it up if it would give her pleasure but she hardly told anything about it when I had been so good. With love, Hetty.”
But the warm feelings between Hetty and Sylvia’s servants rarely lasted for long. One night, while visiting Sylvia, Hetty told Fally Brownell to “take your duds and clear out!”
Hetty said, “I don’t want you here! My aunt doesn’t want you here either. You should just go away.”
“I won’t,” Fally said. “Not till I see Miss Howland and talk to her. Miss Howland hired me. You don’t have the right to turn me away.”
The antagonism between the two came to a head one chilly morning in February or March of 1862, when Aunt Sylvia, Electa, and Hetty were eating breakfast downstairs. Fally was upstairs, straightening the rooms and preparing to fill the water pitchers. Hetty excused herself from the breakfast table and went upstairs. A few moments later, Aunt Sylvia and Electa heard crashing and banging sounds.
Aunt Sylvia looked up from her plate. “What’s that?”
Electa sighed. “Miss Robinson. She makes all sorts of noises.”
“Go and see,” Aunt Sylvia said.
In the entry, at the bottom of the stairs, Electa found Fally Brownell, disheveled, shaken, her clothing torn.
“She pushed me, and I fell down the stairs!” Fally cried. Hetty charged down the stairs, silently seething while Fally relayed the story to Electa and Aunt Sylvia.
“I can’t have such goings-on in my house,” Sylvia said. She sent a servant to fetch Thomas Mandell, a partner in Isaac Howland Jr. and Company and perhaps her most trusted advisor. When the servant left on his mission, Hetty’s mood abruptly changed.
“Please, Aunt, don’t tell Mr. Mandell about this. He doesn’t have to know anything about it. Please. It will ruin my character to have this go out around town!” Sylvia reluctantly agreed and sent word to Mandell not to come, after all. But one night not long thereafter, as she lay in bed, she turned to Electa Montague and said of her niece, “How could she be so cruel and treat me so, when I have done everything in my power to make her happy?”