I think it's the same reason the Rodgers and Hammerstein model really only lasted a couple generations. The fallacy of naturalism.
The big, enduring flaw in the Rodgers and Hammerstein model of musical theatre -- the thing George M. Cohan and George Abbott knew -- is that there's just no escaping the fact that a musical isn't real, and there's no good reason to pretend that it is. R&H abandoned musical comedy and instead followed the model of the modern social realism play, counting on their audiences to "suspend disbelief" so completely that actors speaking naturalistic dialogue, in naturalistic costumes, in front of naturalistic sets, could suddenly break into song backed by a 24-piece orchestra, and nobody would flinch. But plays and musicals are so different in so many ways, and trying to bend the conventions of plays to accommodate the musical form created an uncomfortable hybrid.
To be fair, the R&H model was very successful for a couple decades, and audiences got used to its weird fake-naturalism. But rock and roll emerged, the teenage demographic appeared out of nowhere, and the R&H audience started getting older and older, without younger audiences to replace them. Now, as the New American Musical takes over the art form, young audiences are flocking back to the musical stage, but they just don't connect to the old-school musicals on a gut level. The pace of storytelling in those old R&H shows is much more leisurely than we're used to today, the morals and the music are old-fashioned, and so are the assumptions about audience expectations.
Movie musicals had the same problem. Movie musicals died at the same time that the R&H musical fell out of mainstream favor in the 60s. Sure, some movie musicals were made after that, but very few were successful. The movie musical was no longer the thriving film genre it had been. Mainstream audiences turned away.
The more films became naturalistic, the harder it was to make movie musicals work, especially if they were based on R&H-style stage musicals. By the late 1960s, musicals and contemporary filmmaking had mostly parted artistic ways, exactly as musicals and popular music had parted ways. Luckily, as the musical theatre dropped out of the mainstream, the art form had the space and freedom to evolve like mad over the next decade, in the hands of Stephen Sondheim, Hal Prince, Michael Bennett, Fred Edd, John Kander, Bob Fosse, and others.
But all the reasons stage musicals didn't succeed at naturalism were amplified on the screen, where everything but the singing seemed completely real. Films were no longer overtly stylized, as it had bee in the 40s and 50s, but musicals still were. They couldn't help it. And nobody understood this problem. It's not that audiences were thinking "Wow, their singing is so un-naturalistic!" But movie musicals just didn't feel right or make sense in the world of the authenticity-centric 1960s.
Some movie musicals solved this problem, and got us back on track to where we are today, now with at least one major movie musical opening almost every year. These films accepted the conventions of the inherently artificial musical form as it is. Frank Oz created an entirely stylized world for Little Shop of Horrors, including the mystical Greek Chorus girls moving in and out of reality, taking the conventions of the MGM movie musical and adapting them for our Age of Irony, finding a style in which breaking out into song wasn't jarring. Rob Marshall set the numbers in Chicago as surreal production numbers swimming around inside Roxie's twisted, damaged, show-biz-infused mind. Nine worked essentially the same way. Marshall's approach is wonderfully in synch with the New American Musical onstage: Admit the artifice. Movies aren't reality.
Level with the audience. Respect their intelligence.
After all, when people say they don't like movie musicals, most of them mean they don't like faux-naturalistic movie musicals like Oklahoma! and The Sound of Music, films in which the style and acting are naturalistic -- except when the music starts. Most people who think they don't like movie musicals have probably never seen Shock Treatment, Hedwig and the Angry Inch, Absolute Beginners, Colma, or Once.
In John Turturro's fascinating experiment Romance and Cigarettes, he fully embraces the conventions of old-school, faux-naturalistic movie musicals, but counts on his audience's cultural expectations to add an ironic, contemporary, meta-layer to every musical number, both expanding the emotional moment and also calling attention to its own inherent phoniness. Woody Allen's Everyone Says I Love You and Steve Martin's Pennies from Heaven work essentially the same way.
Another solution is the diegetic movie musical, in which the act of making music is part of the action of the story, rather than just the language of the storytelling. In Cabaret, Fosse turned all the songs into actual performances inside the story, so that the songs became both naturalistic and Brechtian. In Hedwig, all the songs are also performances.
And a few movie musicals, like Colma, Absolute Beginners, Repo, Across the Universe, and the recent short film Zombie Musical, have rejected almost all the other movie musical rules -- including the diegetic "work-around" -- and instead adapted the conventions of music videos into new rules for feature films. As today's stage musicals are doing, these film musicals don't try to find new approaches within the old rules, they create new rules.
It's ironic that so many music videos on MTV, particularly back at the beginning, borrowed a lot from the great MGM movie musicals, and now movie musicals are borrowing from MTV. I remember when the brilliant Moulin Rouge was released, it seemed to me like a musical Arthur Freed would have made at MGM in the 50s, if they'd had the tools we have now. But much of it also felt like really amazing, big-budget music videos. I think we'll see this MTV trend increase as the next generation starts making movie musicals, and as the ever decreasing cost of making films today democratizes filmmaking even further. And that's really exciting.
As we might expect to happen as the film musical evolves, the film Once combines forms. First, it is a diegetic movie musical, in which the act of singing and playing music is part of the action of the plot, so the film works in a kind of naturalism. But it's also an MTV movie musical, with many of its songs set to montages, a time-telescoping device taken from the non-musical romantic comedy. And like David Byrne's movie musical True Stories, the songs in Once never further the plot; instead they amplify or expand on the emotional or thematic content. Exactly, in fact, like Spring Awakening does, whose movie version is in the works. And also Stephen Sondheim's Company -- I would love to see Rob Marshall make Company into a film.
But no doubt about it, the movie musical is back. And, like its stage counterpart, it's more interesting, more adventurous, and more vigorous than ever before. Just think about all the cool movie musicals released in recent years -- Nine, Repo, Colma, Once, Fruit Fly, The Devil's Carnival, Across the Universe, Hairspray, Dreamgirls, Hedwig and the Angry Inch, Chicago -- and we have to include High School Musical, which did a lot in bringing a young audience back to the musical theatre. And all these films are so different from each other! So much original work is going on. The movie musical as an art form is evolving and moving into the twenty-first century, and that's very cool.
Long Live the (Movie) Musical!