A schema is your model of the world, it defines what your annotators can and can not express. Some schemas are very clear, such as “person”, “city”, “company” while others can be quite vauge such as “negative sentiment”, “expression of happyness”.
As a manager of an annotation project, it is your job to define a schema that both captures the phenomena that you want to capture while also being easy for you annotators to understand and recognize.
When starting an annotation project with a newly defined schema, it is worthwile to do a test run and test for tag agreement with LightTags Analytics module. A good schema will show that your annotators consistently agree on a tag. That is when one annotator said “A” the others also said “A”.
- Capturing these mistakes early on in an annotation project gives you the opportunity to correct early rather than
- collect a large set of inconsistent or imprecise annotations.
A schema has just one attribute, it’s name. If you have multiple schemas the names must be unique.