On March 18, when the Metropolitan Museum of Art opens an annex at Madison Avenue and 75th Street in Manhattan, it will be attempting to shrug off the ghost of a museum past.

The specter is the Whitney Museum of American Art, which called the iconic Marcel Breuer building on that corner home for nearly five decades. In an eight-year deal, the Met is leasing the Breuer building from the Whitney— which relocated to its dazzling new Renzo Piano–designed home last year—in an attempt to expand its Modern and contemporary-art presence.