How we did it
Industry selection and scope
It probably started from a conversation with the annual reporting team of a large company. An IR officer said: “Of course we like to see our report rated and ranked among the best, but what matters most to us is to lead in our industry, against our peers. Ranking at, say, No. 50, even among hundreds of annuals, is appreciated, though less important than coming up with a top 5, 3, or, naturally, No. 1 within our industry.”
This year, for the first time, the ReportWatch monitoring process wasn’t kicked off with a list of companies but started with the selection of a number of major industries, many of them often recognized for their contribution to worldwide wealth and growth. To give but a few examples:
- as a whole, the food and beverage industry ranges between 16% and 20% of the world economy;
- the global construction sector represents about 15% of global GDP (with $100 trillion to be spent over the next ten years);
- more than 50 million jobs are owed -directly or’ indirectly- to the automobile industry;
- the chemical industry accounts for 2/3 of EU manufacturing surplus;
- out of every $100 spent, telecommunications receive $2.50;
- and so on.
Worth a (change of) focus, isn’t it? To be honest, that new industry-based approach has long been advocated by investors, stakeholders, and analysts using the Annual Report on Annual Reports as a reference for report comparison. As to report makers themselves, how many don’t start their job -and evaluation- by looking at their direct competitors as a first step -or reflex response? (See the quote above). We had already set that competitive approach in motion with the Compare column/line in past rankings.
Why those industries and not others? Some, if not all, are reputed for better reporting practices. Furthermore, we were limited by time and resources.
The scope was an issue, too. How and where to set the boundaries? What is rather easy for airlines or telecoms is less obvious for chemicals and the automobile industry. As to the former, how to know where to draw a line for an industry whose output is made of more than 70,000 products? We decided to focus on a number of industry leaders. As regards the latter, we have focused on car makers. Why? Well, why not. Considering the whole automotive world, i.e. adding bikes (with or without motor) might of course make sense, but then how to deal with trucks, buses and coaches, tractors, and equipment manufacturers? In this and in other industries (e.g. construction, where players more focused on infrastructure were left out), the idea was to come up with meaningful comparisons and manageable peer groups.
Logically, we dropped the idea of publishing a consolidated ranking as we did all these last years. This would have brought very little value added, compared with the depth of the breadth and depth of research used for our industry rankings.
Focus on listed companies
Our sources for selecting industry players included published international and national company rankings as well as long-running ReportWatch internal desk research based on company positions in their industry, peer group benchmarking, and past annual reporting records. Though still imperfect -owing to various factors such as lower reporting standards, emerging reporting practices, less developed IR policies, a deficit in report entries- our lists of companies and their reports make up a relatively representative cross section which reflects the industrial, geographical and stock market diversity upstream, and best reporting practice downstream. In the wake of the power shift in the global economy, the number of reports from now somewhat improperly named “emerging markets” has increased significantly over the past few years.
There are hundreds of thousands -or even millions- of institutions in the world releasing yearly reports, and some of these can be very exciting -a number being even much more compelling than compliant or puff pieces from the private sector. Still, it seems that the annual report was primarily invented for listed corporations to report to their shareholders. Even though U.S. Steel is often referred to as the first certified annual report, published in 1903, "in 1959, IBM hired Paul Rand, a prominent book designer, to create its annual report. As a result, the high-concept annual report was born." (according to the Addison Annual Report Handbook 2005). Since its inception the Annual Report on Annual Reports has therefore focused on reports from listed firms. These can be compared more easily, and this is even more the case due to the effects of globalization (including unwelcome standardization), permanent access to information channels (and internet), and the extensive application of international accounting and reporting standards.
However, in many countries and industries, non- or partly-listed companies -privately or government-owned- play an important role. Moreover, some of them are known for high-quality reporting practices. Our survey has always left the doors open to those companies.
Note. Reports for a fiscal year ending any time in the year 2016 (March 2017 at the latest) were considered.
Were not considered for selection:
- Reports for a fiscal year before or after 2016 or half-year/quarterly reports;
- Financial sector companies (banks, insurance, investment funds, financial holdings);
- Privately owned companies (except those electing to compete);
- Purely government-owned companies (except those electing to compete or those compared with);
- Wholly-owned subsidiaries (except those electing to compete);
- Investment, income, mutual or real estate funds and trusts;
- Listed stock exchanges;
- Central banks;
- Development or reconstruction banks and similar financial institutions;
- Public agencies;
- Non-profit organizations from any sector.
Note. The name of the company that appears in the Annual Report on Annual Reports is the one as referred to on the covers or as written or abbreviated in key report sections (profile, message). For legibility reasons, legal forms or words such as corporation, company, group, holding, etc. are not reproduced. Except for clarity and communication, names do not take into account mergers, acquisitions or brand identity changes that might have occurred and been approved after the fiscal year-end or the report release.
Report evaluation criteria
ReportWatch evaluation criteria have consistently been based on a well-balanced perspective blending financial and operational analysis; short- and long-term performance aspects; strategy and operations; visual and textual elements; share- and broader stake-holders’ issues; information content and communication style –whether in print or online. Report assessment criteria have to cope with trends and challenges. They have evolved and are updated and upgraded regularly.
For the annual reports for a fiscal year ended in 2016 five reporting areas were evaluated:
- Communication & style;
- Operations & sustainability;
- Strategy & leadership;
- Figures & financials;
- Investors & governance.
The following report items were scanned and scored:
Online report and navigation – Cover and introduction – Profile/Overview – Key figures – Message, identity, branding – Key performance indicators – Business model, value and strategy – Outlook – Goals and targets – Business report – Financial review and management discussion – Risks – Investor information – Share items – Sustainability objectives and integration – Social and environmental impact – Leadership and management details – Governance – Compensation – Statements and notes – Charts, diagrams, maps – Style, layout and design.
Industry and company specifics
In parallel with the report scanning and scoring, we paid special attention to industry and company specifics and the way some were reported. Cases in point: the wave of mergers in agribusiness and chemicals, pipeline development in pharmaceutical companies, portfolio valuation and EPRA measures in the property sector, food waste at retailers, emissions in the automobile industry in general and at some firms in particular, corporate responsibility across the board, to name but a few.
The addition, or, better said, the combination of report evaluation criteria and industry or company peculiarities required teamwork of financial analysts, industry experts, investor relations specialists, corporate communication advisers, accountants, economists, copywriters…
A number of distinctive features, deficiencies, positive aspects, oddities, originalities, absurdities, etc. (you may choose other adjectives, of course) were noticed and are reported for some, allowing our readers to check best, good and bad or worst practices.
Report rating scale
Annual reports are first listed in alphabetical order and then ranked by grade.
Comparison and benchmarking
A bar chart shows how annual reports in an industry compare on the five sets of evaluation criteria.
For some a cylinder-based chart is included to set forth the weight of each set of criteria in the total report score of each company.
Those charts are helpful to grasp best practice in the reporting areas.