Reliability in Research

Oh goody, blogs are starting again, what a thrilling way to start February…

I thought I’d start with an easy blog topic to not frazzle my mind completely on this lovely Sunday morning!

Reliability is a way of kind of knowing what to expect from something/someone/a researcher or any area of research in psychology in this case. If a researcher is able to produce the same quality of work and similar results, they are considered reliable. Reliability is built upon consistency as well as repeatability and any researcher should consider reliability when designing a study. If results from an experiment have reliability then it is more likely that if the study is replicated then similar results will be found, therefore building upon the big stirring pot of psychological knowledge so that one day, the research can go further and something can be discovered which could change the world as we know it, (cheesy i know).

Reliability is not just seen in research, it is also seen in daily life. For example, in sport, a football player must have a reputation of being good in order to be popular among the fans. They must have a reputation of having a high level of playing performance over a range of situations, in this case, different games in different places etc., and if this player is reliable then his behaviour would be expected to be consistent and good every single time he played a game. Also, his reputation would increase of being a good player if every time he played; he got better and better, therefore reinforcing the fact that he a reliable player.

Joppe (2000) defines reliability as… ‘the extent to which results are consistent over time and an accurate representation of the total population under study is referred to as reliability and if the results of a study can be reproduced under a similar methodology, then the research instrument is considered to be reliable’.

However, although reliability is essential in research, a measure can be perfectly reliable but not valid. For example, say the weighing scales in your bathroom say that you weigh 2 stone less than your actual true weight and every time you weigh yourself, this same figure occurs. Although this scale is very invalid, it is perfectly reliable as your weight is consistently the same every time you weigh yourself. Another example involving research would be a questionnaire assessing job interviewees and whether they are right for the job they are applying for, including questions like ‘do you eat pizza when you watch TV on a Saturday night?’, although this question is invalid on assessing whether an individual is right for a job, you would most probably get similar answers when testing the same sort of individual, therefore making it reliable. This could suggest that although it is ‘an essential’ to have reliability when conducting a research study, it could not work with its nice companion ‘validity’. A research experiment has to be valid and reliable in order to be completely accurate.

Overall, reliability is essential to have for any researcher under taking a research experiment as it shows that the findings are more likely to be replicated and therefore it can be built on to find even more significant results. However it is inevitable that research could be reliable but not valid as seen in the examples I have given so for an experiment to be able to have reliable findings, they have to also make sense and have validity.

Advertisements

About psuc9d

2nd year psychology student at bangor university

17 responses to “Reliability in Research

  1. Great blog! To add another example of the importance of reliability in an experiment is provided by the cold fusion case, of 1989.

    Fleischmann and Pons announced to the world that they had managed to generate heat at normal temperatures, instead of the huge and expensive tori used in most research into nuclear fusion.

    However researchers all over the world were quite simply astounded by the claims made by Fleischmann and Pons, and so many attempted to replicate the experiment, to no avail. Whether the researchers lied, or genuinely made a mistake is unclear, but their results were clearly unreliable.

  2. Pingback: Homework for my TA (Week 3, Semester 2) | psucab

  3. psuc28

    While you said you will start with an easy subject, it’s clear the subject is easy for you because you have a good understanding of Reliability. You could have maybe included different types of testing reliability methods such as equivalent forms, the split half method and the test-retest method (further information on these topics can be found in my previous blog post- (http://psuc28.wordpress.com/2011/10/13/reliability-vs-validity/).

    You touched upon the effect of something being reliable but not valid and I like your example of the effect of reliability of scales saying you are 2 stone lighter, consistently so it is reliable but it is not valid because they are wrong, (I think many would like to believe it is valid!). It is a good real word example and another example I used in my previous blog post also illustrates this-

    “Joe walks into an ice cream shop every night at 5 pm and orders a milkshake. Every night at about 5:05 pm, the soda clerk gives him a milkshake that tastes just like it did the night before. Joe loves this because he knows he can walk into this ice cream shop and always get a drink he likes that tastes the same way each time. One day Joe invites a friend from work to join him. They both go into the shop and order what Joe thinks is the reliable milkshake. When it is delivered, however, his friend from work tastes the drink and declares, “This isn’t a milkshake-it’s an ice cream soda.” (Buelow & Hinkle, 2008)

    This explains how although Joe reliably received what he believed was a milkshake every day it was however a cream soda so the statement he received a milkshake every night was not valid. Showing that you are correct just because something is reliable it does not have to be valid.

    ————–

    Reference

    Beulow, M, J. & Hinckle, L, J (2008) Journal of Neuroscience Nursing. Research Corner-Why Are Reliability and Validity Important to Neuroscience Nurses? Volume 40 (6). Pages 369 – 372.

  4. Pingback: Blog Comments For My TA « All About Psychology

  5. You’re blog is really good and it’s very clear that you know what you are talking about. I would suggest that you could perhaps explain the different types of reliability in more detail. So for example you could introduce terms such as inter-rater reliability which involves looking at how the responses of different researchers relate to each other, and therefore whether or not the methods of collecting the data are in fact reliable. This could be improved by making sure that the observers are all well trained previous to the experiment and that a clear way of measurement is being used.

  6. Pingback: Homework for TA :) | Do you need statistics to understand your data?

  7. Although it is clear you totally understand the concept of reliability and the importance of it in research your argument could have been made stronger by giving examples of research where the reliability has been incredibly low. Without this the argument feels a little unsupported but other than that your blog was good and interesting to read.

    I talked about reliability in a previous blog and when researching studies with low reliability I found the study conducted by Andrew Wakefield into the MMR vaccine (http://briandeer.com/mmr-lancet.htm). This study contained shockingly low reliability as when the experiment was repeated no significant link between the vaccine and autism was found.I’m sure most people have heard that this study had disastrous implications in the real world including the first death from Measles in 14 years.

    You can see from my example that reliability is important and including something like that may have strengthened your (already very strong) argument. You also make some good references to the importance of validity, perhaps you could blog about that next?

  8. As you have pointed out having reliable results is extremely important when conducting research. It is important that researcher not only find consistent results over time but also internal consistency reliability, where the items within a measure show consistency over time .For example if researchers were using a personality questionnaire using a likert scale, the items in the questionnaire should also be reliable in measuring the scale. For example are the questions used to measure openess showing reliable results, and if not the researcher should consider removing that scale (Rammstedt & John 2007).

    A way to test internal consistent reliability is by using the statistical test Crombach alpha (Crombach 1951). If items show a score of .7 or more then it is considered to show good internal consistency. Also shown on an spss output of Crombach alpha is the correlation between the items, if items have a correlation less than .3 then the researcher should consider removing the item as it is not a reliable measure.

    Finding reliable results is extremely important as without it researchers would be unable to formulate theories, draw conclusions and generalize their findings, and without any of these the research is pretty pointless.

  9. Pingback: Blog Comments- Semester 2 « All About Psychology

  10. Hello
    Great blog, really nice to read and easy to follow, you’ve explained the importance of reliability in research and you have also mentioned how it doesn’t work on it’s own, findings need to be valid too 🙂
    One other aspect you could have added in is how researchers can actually assess the reliability of studies. For example the test- retest method that involves the same test being administered at two different points in time. Marsh, Sullivan and Parker (1999) used this in their study regarding a measure of anxiety for children to assess whether the measure was successful or not. 🙂

  11. Pingback: Comments for Sir Alex Currie « psuc97

  12. Pingback: Homework for my lovely TA :) | serenapsychology

  13. Good blog!!! It was a good read even if it is about stats 🙂 I loved the real life situations you used especially the scales one I would my scales I was 2 stone lighter 🙂

    According to Miller there are three aspects of reliability; equivalence, stability and internal consistency. Equivalence refers to the amount of agreement between two or more instruments that are administered at nearly the same point in time. This is measured through a parallel forms procedure in which one administers alternative forms of the same measure to either the same group or different group of respondents.

    With internal consistency this is especially important when working as part as a team as inter-rater reliability which refers to the consistency with which the observers make judgements is needed to create a valid study.

    In conclusion, basically we need reliability and we need to write these lovely blogs.

    Happy Blogging 🙂

  14. Pingback: Comments for my TA « leprice91

  15. Pingback: Comments 1 Semester 2 « Bloggenrolla

  16. Really good blog this week. I have covered this topic before and found that reliablity is extremely important. Its important within the world but also within sciences and psychology. For example in sciences people results need to reliable each time, humans aren’t robots so we need to make sure that we repeat results in order to make sure our experiments are valid aswell. I like the examples you have used aswell, and with psychology you need to make sure your experiments and your results are reliable. In order to be regarded as high up in psychology this needs to occur because if someone comes along and conducts the experiment better they might get different results and disregard your experiment. This will not look good if this happens and is extremely unprofessional.

  17. Pingback: Comments for this week 10/02/12 « psucc8

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: