Overview

For this assignment, you are asked to write a report on the activities covered in the notebook for Topic 7 and Topic 8.

In particular, your report should consist of a Jupyter notebook containing the following four sections (further details regarding the content of each section are given below).

Section 1 :
A report on what you discovered while undertaking an assessment of the accuracy of spaCys entity extraction capability on a personalised sample of sentences.

Section 2 :
A report on your gender classifier.

Section 3 :
A report on your code that extracted feature sets for the characters in a novel.

Section 4 :
A report on your investigation into differences in the way male and female characters are portrayed.

Details of Requirements

Your submission will be marked out of 100. Please read the following guidelines carefully.

Section 1: Assessing spaCys Entity Extraction Capactiy

There are 20 marks available for this section.

• You should run the entity extractor on a sample of 100 sentences and report on how accurately it has been performed.
• The sample you use must be produced using your candidate number to generate a personalised set of sentences for this evaluation.
• You do not need to describe your assessement of each sentence individually, but you should give overall statistics and use illustative examples from your sample.
• Your analysis should be broken down by the type of named entity.
• When an error occurs you should describe the nature of the error. You should distinguish the following cases
  • where the wrong entity type is assigned to a span
  • where the wrong span is identified
  • where an entity is missed altogether
• A confusion matrix can be used here to summarise what you have found.

Section 2: Gender Classifier

There are 20 marks available for this section.

• You should present the code of your gender classifier and explain how it works.
• You should use the names.csv data or, if you prefer, some other comparable source of information about the gender of names.
• Your code should deal with cases where a character is referred to by more than just their first name (e.g. John Jones).
• Your code should deal with cases where a character is referred to using a title (e.g. Mrs Smith).
• By running your gender classifier on a sample of data and reviewing the results, provide an indication of how accurate your gender classifier is. What proportion of names are being correctly analysed?
• Bonus marks will be awarded if you successfully deal with situations where just a surname is used (e.g. Smith) after the gender of that character has been revealed (e.g. Mrs Smith).

Section 3: Building feature sets that characterise the way a character is portrayed

There are 30 marks available for this section.

• You should explore a number of alternative ways of characterising the way a person in portrayed by a novelist. This should include implementing the suggestions that appear in the Topic 8 notebook. You are encouraged to go beyond the suggestions made in the Topic 7 notebook.
• You should describe the code that you have written to create feature sets for characters.
• You should describe the code that showned how you were able to extract features in situations where one of the pronouns he, she, his and her is used in a novel.

Section 4: Investigating differences in the way genders are portrayed

There are 30 marks available for this section.

• You should make it clear how you have aggregated feature sets across the male and female characters appearing in at least two collection of novels. These collections could be novels by different authors, different sets of authors, or sets of novels that were written at different periods in history.
• You should discuss the result of measuring the cosine similarity of the aggregated male an female feature sets. The reason to consider different sets of novels is to look at differences in gender-based cosine similarity in the different collections.
• In Section 3 you should have considered a number of alternative ways of deriving feature sets for characters. In this section, you should present the results of using these alternative approaches.
• You should explain what you have done to assess the cosine similarity of pairs of features sets that are aggregated over randomly selected characters (i.e. characters that arent split up on the basis of gender). This should provide an indication as to whether the differences you find when making a gender-based comparison are meaningful.
• You should explain how you went about assessing what the impact would be of an imbalance in the number of male and female characters. Is there an gender imbalance in your gender-based comparison.