Hospital signs and NHS leaflets are also highlighted as not always being clear. The report gives an example of a patient told to go to a hospital for a chest X-ray, but the signs said 'radiology' not X-ray.
The report says common terms like 'heart section' may be easier to understand than 'cardiology department'.
The term 'chronic', meaning long-term or persistent is often misunderstood as meaning serious.
One review found that patients often retain around half of the information they are given in a doctor's consultation, and only half of what they do remember is generally correct.
Understanding what doctors say is called 'health literacy', and the report says this varies across the UK.
Health literacy was highlighted as being low in London, and among older people, some ethnic minority groups and people on lower incomes.
The Health Literacy report says GPs and hospitals need to make information clearer.
It suggests doctors should check what a patient has understood and remembered by getting them to repeat back critical information in their own words.
In a statement, Maureen Baker, chair of the Royal College of GPs says: "Too often, our healthcare environments fail to recognise the needs of people with different levels of understanding about their health, meaning that patients are failing to receive the right care at the right time.
"We know that low health literacy affects all areas of health and health care, which why we want to encourage GPs and the wider NHS to ensure they are communicating complex information in a clear and manageable way."
Commenting on the report, Joyce Robins, co-director of the patient group Patient Concern, told us consultations can be rushed: "Doctors must explain carefully the implications of conditions to patients, but often there is just not time for this.
“You can be bemused after your couple of minutes in the surgery. So seldom do you have something clear and concise pushed into your hand."
To provide even greater transparency and choice, we are working on a number of other cookie-related enhancements. More information