Elections Commission second media report lacks reliability By Prem Misir, Ph.D. Guyana Chronicle February 11, 2001 SHEER arrogance is exhibited in the way the methodology continues to be presented! Some of my criticisms of the Media Monitoring Unit (MMU)'s previous report pertain to the vague methodology presented. In fact, I alluded to the term 'methodological adventurism', staged by the MMU. In its second report, there has been little evidence of any improvement in its methodology, and herein lies the arrogance. The MMU's second report brings a partisan and telling dimension to Guyana's sagging journalistic life. In many respects, the MMU is intended to breathe some fundamental fairness, accuracy, and balance into the hellish and inappropriate brand of journalism as practiced today in this country. And there are many people who see the MMU report as impacting Guyana's premature journalism. While we continue to revel, however, in some of its findings, let's also understand and assimilate how these results emerged. Problems with methodology Serious problems with the methodology still prevail! The report only focused on news stories. What we don't know is whether all news stories during the monitoring period were included, or whether a sample was taken. The methodology must be comprehensive, answering the following questions: Who were the subjects? Were inclusion and exclusion criteria indicated? How were the news stories selected? How many news stories were studied? What research design was used? Were instruments (forms) used? Did the team document the reliability and validity of these instruments? Were operational definitions given? Were threats to internal validity controlled? What data analysis procedures were applied? A few of these questions already are answered, but there are others that need to be addressed. GECOM Media Consultant Robert Norris, in responding to several methodological issues, maintains that anyone can visit GECOM and review the MMU team's work. This is fine. But the MMU report is a public document that must fulfill the parameters of a research record. Keep in mind that these reports eventually will become historical documents. The MMU report uses content analysis which is intended to produce an objective and systematic account of news coverage. The MMU team did not achieve this goal because we need to have transparent and explicit rules and procedures to reduce the monitors' subjective predispositions. If the team used these rules and procedures, then the report demonstrates vagueness on this front. Using subjective ratings to classify news stories as positive, neutral, and negative, does contain systematic measurement errors. For instance, systematic errors can emerge if monitors include concepts external to the construct in their judgements. Suppose that the age of a monitor, or his/her religion influences the ratings on news stories. Well, although age and religion can shape news stories, in the final analysis, they may not have been included as part of the theoretical definition, and therefore, are external to the concept of news stories. But since many of these concepts are difficult to objectively measure, we need to find out the determinants of systematic errors, in order to reduce their impact on the study. What could partially assist in reducing systematic errors is the use of inter and intrarater reliability. Again, the MMU report did not utilise rater reliability for the news stories. Interrater reliability would demonstrate the variation between two or more monitors who measure the same group of news stories. Intrarater reliability would show stability of the news stories measured by one monitor on two or more occasions. The application of rater reliability is really needed. Intrarater reliability is the reliability between measurements made by the same person (rater, coder, or monitor) on two different occasions. Interrater reliability is the reliability between measurements made by two different monitors (raters, coders). In practice, the intrarater reliability has to be administered prior to the interrater reliability. Apportioning 'positive', 'negative', and 'neutral' labels to entire news stories, is subjective, and ignores the nuances in each news story, according to the CMPA. And I agree that these positive, negative, and neutral measures are inherently subjective for reasons beyond our control. But what may help to ensure that the news items are reliably labeled as positive, negative, and neutral, may involve classifying news stories into event or issue-focused news, and into topics. The methods section has a few definitions for 'Government', 'GECOM', and 'General', but none of these is operationalised for measurement purposes. Again, the concepts 'positive', 'negative', and 'neutral' are only conceptualised in terms of the question, "is this story likely to lead a viewer, listener or reader to a positive or negative impression of the subject of the story?" They have not been defined in operational terms. It was never my intention, initially, to review the methodology in such detail. But I felt compelled to do so because of the historical import of this document. The issues raised may necessitate further discussion, given the national significance of the MMU report. But for now, let's remove ourselves from the 'methods' issues, and focus on other areas of the MMU report. Training of Monitors Reliability is significant to validity in any study. The data may not be interpreted with confidence, unless, in this case, monitors, who collect, record, and reduce the data are reliable. The monitors (referred to as 'raters' in other studies) should have some intensive training to ensure that procedures are standardized. This training is very important, especially when the procedures are new or unfamiliar, or when subjective observations are applied. The Center for Media and Public Affairs (CMPA) requires research assistants on content analysis projects to be exposed to 150 through 200 hours of training, assimilating rules for coding and categories, inter alia, prior to begin work on a project. In fact, during training, the CPMA ensures that researchers code stories, and then their work is compared to the work of earlier coders (also referred to as research assistants) until a minimum reliability level of 80 per cent is attained for all variables. New coders must reach agreement as the previous coders at a minimum of four out of five times, according to CPMA. The MMU 's methods section indicates that monitors receive training, but with no specifics. Guyana Chronicle The Guyana Chronicle (GC), the report claims, does not offer some coverage to a wider range of parties as the Stabroek News (SN) does. The two newspapers may have a different ratio of advertisements to news stories, both competing for space. If, for instance, GC has a higher ratio, then it will have less space devoted to news stories. Further, GC does carry information on some nation building projects, periodically; this coverage also will create a paucity on space. Given this intense competition for space, nevertheless, GC does have some coverage of several political parties in any defined time period. GC's headline that the High Court 'upheld' the results of the 1997 elections also was criticised for not carrying the different dimensions of the Court's decision. However, this story sprawled on the front page and page two, despite its caption, covered as much ground as SN, on the High Court's decision. Nevertheless, coverage of stories depends upon the raw materials carried by the reporter at the time, and based on the reporter's personal construct, inter alia, this was the story that he assimilated as it happened. If monitors reexamine this story, they will find that it had sufficient details to facilitate an informed judgement on the High Court's decision. GC does not have to carry all stories pertaining to the High Court's decision. Editorial prerogative kicks in at this point to decide on a qualitative rather than a quantitative presentation. Monitors cannot make a judgement on the newspapers, in reporting that they 'hardly answered all the questions raised by the ruling...' This type of value judgement is inappropriate for monitors detecting media bias; stay with the factual materials. Guyana Broadcasting Corporation The report indicated that the Guyana Broadcasting Corporation (GBC) should have verified President Jagdeo's statement that People's National Congress (PNC) member Winston Murray, while in the National Assembly, stated that he unreservedly endorsed the voter ID card legislation. The verification via Winston Murray was unnecessary because the contents of Parliamentary Sessions are documented in the Order Paper, a public record. Again, GBC was castigated for providing minimum coverage of the PNC rally. The MMU report asserted that although GBC aired a story about the PNC campaign launching, minimum coverage was apportioned the rally, and greater treatment given to the PNC Leader's attack on Dr. Kenny Anthony. The PNC Leader's attack on Dr. Anthony occurred at the rally. GBC news editors, in their judgement, believed that Dr. Anthony's responsibility for justice and governance among CARICOM Heads of Government had greater news value for Guyana's fragile democracy than a political rally. I don't know what the hullabaloo is with respect to GBC's rejection of the PNC's advertisement. GBC did not carry the advertisement because it was 'factually incorrect'. Any media house has the right to reject an ad, providing reasons for the rejection, as stipulated in the Media Code of Conduct. The falsity of this ad could lead to public disorder, as I pointed out via the Media Code of Conduct. GBC's position, therefore, on this matter is quite consistent with mine. Guyana Television The MMU report said that GTV aired information on the High Court's decision on January 15. The reporter indicated that both sides felt that they won. Senior Counsel Ralph Ramkarran's views on the matter were carried. However, the report claimed that no PNC spokesperson was given the opportunity to present views. GTV management in response said that Senior Counsel Ramkarran was solicited for his views in his capacity as legal counsel for the defendant. At that time, GTV tried to engage Senior Counsel Rex McKay in a banter with Ramkarran, but with no success. Profile of Monitors Given the usual political sensitivities in the election season, the MMU should consider giving all media houses a profile of the monitors. Further, none of the monitors in the MMU should be allowed to hold office in any political party or campaigning group. These are reasonable requests to which the MMU can accede. Government I say again that a government does not contest an election, a political party does. In a democracy, a party wins an election, sets up a government to implement its mandate. This mandate loses its purity through parliamentary discourses and the way that parliament votes on legislation, for the good of all the people, not merely 'party' people. Generally, a government involves political processes that produce authoritative formulating of rules and policies that are binding and pervasive for all in society. A democratic government because it represents a nation, then, enjoys a status above elections. Therefore, the MMU's decision to see activities of this government, as election related, is a travesty and a violation of the democratic norms that shape a government's functioning and being. This approach is inappropriate, as it means, then, that this Government's policies, programs, and projects are electioneering materials and gimmicks. I would urge the MMU to rethink its position on this matter. The first two MMU reports, with all their shortcomings, are now historical documents. Future MMU reports, however, must comply with scientific ingredients for observing and reserving the sanctity of their place in Guyanese history!! Follow the goings-on in Guyana in Guyana Today If you do not see the 'Guyana: Land of Six Peoples' drop-down menu on the left then please click here to enter through the front door which will give you additional options and information.