Calculating intercoder reliability

When several researchers code the same text using the same scheme with defined codes, we would expect them to produce similar results. Calculating intercoder reliability has become a well-established procedure to safeguard consistent and intersubjective application of a developed coding scheme, with respective scores serving as a quality criterion for the reliability of the coding process.

Deen Freelon has created ReCal, a great online tool that researchers can use to calculate reliability in multi-coder content analysis. It works with different measurement levels (nominal, ordinal, interval, or ratio-level) as well as with different numbers of coders and – most recently – even with missing data. Thank you, Deen, for providing this highly valuable tool!

1 Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s