E-mail is dying out! Or, at least, that’s if recent research is to be believed. Carol-Ann Morgan, however, in this week’s Thursday Night Insight, reminds us that we can’t always take research findings at face value.
Recently published research tells us that e-mail will be defunct in 10 years’ time in favour of social networking sites, texting and other instant messaging options, and that 98% of people over the age of 65 use e-mail regularly.
I am, at this moment, thinking of all the people I know over the age of 65, and I cannot get these facts past my own “common sense” test. I am wondering if I am in contact with an obscure group of older people who live in some kind of time (or technology) warp, as many of those I know do not even own a computer let alone subscribe to broadband!
So, back to the research. I read on and note that the research was sponsored by a leading broadband supplier and I start to wonder if the research has indeed been conducted using a methodology and a sample which have had some influence on these results. No such sample information is contained in the short article printed in the press. However, it highlights the importance of meaningful and realistic critique of research findings.
In academic research, research critique will centre around the evaluation of reliability and validity for quantitative research, and truthfulness and ‘generalisability’ of qualitative research. However, the starting point of critique, I find, is passing the “common sense” test. As a market researcher, it is one of the first tests applied, and if, on first look, it does not pass this test, it is time for greater scrutiny; of method, of data integrity, of sample. Very little research throws up information which is truly shocking or outside of what we qualitatively observe. The actual figures may be surprising (higher or lower than we thought), but generally, it has to be “believable”; in the framework of experience and understanding for the majority of people.
After this acid test, I tend to use my own research “A” ratings:
- Appropriateness
- Authenticity
- Audience
- Applicability
Firstly, I look at the appropriateness of the research approach to address the research aims and objectives. Here, I am essentially looking at whether the best research design has been employed to answer the research questions, be they centred on understanding, exploration and/or measurement. It can be the case that the researcher has made assumptions about the market and thus designed questions which cannot get to the root of the problem, or where the issues are insufficiently well known to design a questionnaire which can measure them.
Appropriateness of method also covers the area of authenticity of the research findings; that is the integrity of the data, and whether this has been compromised by method or by any other factors e.g. sponsors, timing, incentives, etc.
A key area to look at is the research audience. Knowledge of the sample characteristics and source is needed; who is the sample made up of, what is the size of the sample, where has it been sourced, does it include key representative groups, etc? Understanding of any bias which has been introduced though sample selection needs to be taken account of in the interpretation of the research findings. For example, in the above quoted research, if the sample has been accessed from the broadband supplier’s customers, or an online panel of people who have signed up for e-surveys, this may have had some influence on the type of responses gained, and this may not be representative of a completely random group of people.
My final point relates to application; the interpretation of the data and the transferability of the research. The above quoted research purports that e-mail is dying out because “people prefer to send short messages reaching many people in one go and there are better ways of doing this than e-mail”. This may be the case for certain applications, e.g. chat. However, the research fails to take account of the business use of e-mail, which my common sense test tells me is where the greatest application of this media is.
It is the researcher’s job to ensure that all research reported gives sufficient information to the reader to be able to consider the context of any research findings, thereby enabling the reader to consider if, and how, the research findings can be generalised. Considered critique is a necessary part of the research process.
