A good time to have your coding checked by other coders is when you have built a stable code system and all codes are defined. This means, this is somewhere in the middle of the coding process. Once a satisfactory ICA coefficient is achieved, the principal investigator has the assurance that his or her codes can be understood and applied by others and can continue to work with the code system.
Click the Analyze tab and select the option Inter-coder Agreement.
Click on the Add Coder button and select two or more coders.
Click on the Add Documents button and select the documents that should be included in the analysis. Only those documents are listed that have been coded by all selected coders.
Click on the Add Semantic Domain button and add one or more codes to the domain. Alternatively, you can drag and drop codes from the Project Explorer or the code list into the semantic domain field.
Repeat this process for each semantic domain.
If you are not sure, what a semantic domain is, you find more information here.
Whether you add semantic domains or only a single code in place of a domain, this has an effect on the type of analysis you can run. If you only add one code per semantic domain, you cannot calculate Cu/cu-alpha coefficients. See Methods for Testing ICA.
After you have added coders, documents and codes, your screen looks similar to what is shown in:
The documents are represented by a line for each coder and code. If you have added multiple documents into the analysis, you see a black vertical line that marks the beginning of the next document.
The quotations for the code are marked on the line in the same color as the coder icon.
To the right of each coder line you see some descriptive statistics: The coder Sabine (blue) has applied the code "activities clubbing / going for a drink" 8 times. The number of characters that are coded with this code are 1057 out of a total of 172.407 characters in all three selected documents, which is 0,6%.The coder Susanne (blue) has applied the code "activities clubbing / going for a drink" 11 times. The number of characters that are coded with this code are 1331 out of a total of 172.407 characters in all three selected documents, which is 0,8%.
If you click on one of the lines, only the quotations of the selected code and coder are shown in the Quotation Reader on the right-hand side.
If you click on the line for the semantic domain, all quotations of the semantic domain are shown in the Quotation Reader.
You can remove coders, codes or documents from the analysis by clicking on the red x.
To calculate ICA, click on the button Agreement Measure in the ribbon and select one of the four methods.
See Methods For Testing ICA for more detail.
Selecting the Krippendorff's Cu /cu-alpha coefficient for the above example, yields the following results: