Research by the organisation, which looked at the views of 2,000 people, found that one in six people had been discriminated against because of their age.
In total, over a third of those questioned said they had faced discrimination when applying for a job or promotion.
Denise Keating, chief executive, Employers Forum on Age said: "Whilst many companies have solid diversity policies, this may not run throughout the company down to individual team level, which is an issue that needs to be addressed."
Almost two thirds of those questioned said most or some of their workmates were similar to them.
Rachel Krys, director of the Inclusive Employers, recently pointed out that workplaces often are more productive if they include both young and older employees.
She suggested that older workers have a "real settling, calming effect", while younger employers can bring fresh ideas.