Why Sir George Martin Was the 'Fifth Beatle': 'They Owed Him So Much,' Says Band Expert

The death of producer Sir George Martin – dubbed the “fifth Beatle” – has fans reflecting on his contribution to music as one of the masters of mixing classic and modern sound.

“He was a very generous, helpful and educated man, musically,” Beatles expert and friend Ray Connolly tells PEOPLE of the legendary producer.

Writer Connolly worked with Martin on a TV series about music and for decades has chronicled the story of the iconic band, which included John Lennon, Paul McCartney, George Harrison and Ringo Starr. (McCartney, 73, and Starr, 75, are the only original members who are still living.)

“He was like a very good teacher. He taught The Beatles things they didn’t know,” Connolly says of the late Martin, who died at 90 this week. “They knew everything there was to know about rock and roll music and nothing about classical music. He knew everything there was to know about classical music, and some jazz and folk.”

Connolly adds: “When they got together, they educated each other. The result was ‘Penny Lane’ or ‘Strawberry Fields Forever,’ which would not have been the same unless they had a very helpful and clever musician.”

Take, for example, the piccolo trumpet featured on “Penny Lane.” Connolly says that Martin would have known who to call to make it happen.

“They would have been a fantastic rock and roll band, but would they have had that extra influence? Unlikely,” Connolly explains. “He made their records so much broader. It stopped being rock and roll quite early on and gave them an opportunity to express themselves in ways they could never have considered possible.”

Connolly continues: “Paul wrote ‘When I’m Sixty-Four’ when he was about 16. But it wouldn’t have sounded the same unless he had met George Martin, who said, ‘Why don’t we make it sound like an old-fashioned song using different instrument?’ That was his great gift. He enriched their records and enriched our lives.”

The Beatles had a short but incredibly impactful career, made more memorable because of the variety of styles of music they explored. “In seven years there was an extraordinary output of work and without George,” Connolly shares. “I doubt it would have been diverse.”

The writer worked alongside Martin on a BBC special, The Rhythm of Life, which focuses on what music has given people. “I asked, ‘Why do we have music?’ His thought was that it was the emotional language that runs alongside our spoken language,” Connolly says. “So we can play a sad tune and it’s understandable in Finnish, Russian or in English.”

So did Martin truly consider himself part of the band? Connolly tells PEOPLE the power producer mostly stuck to his role as mentor. “They didn’t hang out with them, it was a professional relationship,” he explains. “All of them exchanged Christmas cards every year. They were fond of him because they owed him so much.”

VIDEO: Paul McCartney Gets Turned Down at Tyga’s Grammy After-Party and Social Media Explodes!

Toward the end of Martin’s life, Connolly wrote a radio play, Sorry, Boys, You’ve Failed the Audition – which aired on the BBC – that imagined what would have happened to the Beatles if Martin had turned them down. Connolly sent him a copy of the show “in case he had missed it.”

“I got a very sad letter back saying, ‘Dear Ray, thank you so much for sending the recording, but unfortunately, I’m stone deaf now and I can’t hear a word of it,'” shares Connolly. “Fortunately, Judy, his wife, had lip read it out to him and he knew what he had said in my play. George couldn’t hear it, which I found really upsetting.”

The admirer chooses to focus not on Martin’s failing health, but his lasting legacy.

Connolly says, “He was like a very good teacher – the sort of person you wish you’d had at school.”