Introduction

Text is a very important part of the data that computers deal with. In this assignment you are going to learn two things: how to process text (the first step in understanding language) and how to write code in small, bite-size portions: functions!

Functionality

Once again you will write your programs in increasingly more powerful pieces. But unlike the first assignment you will only produce two main programs and a number of functions. The number of functions that you write correctly will determine your grade.

Remember the wcount.c program in the textbook and in class. You will find a related program in the file textRW.c. In the original program, the text was read in a character at a time. The program textRW.c reads in text, one line at a time. The program assumes that no line contains more than 500 characters. Your code will not be tested on text that violates this constraint.

To test out textRW do the following:

$ make textRW $ cat verneTest.txt | ./textRW

Produce the following functions that will work on a single line of text:

int chop ( char *line )

• Remove the
  from the end of the line.
• Returns 0 if successful and -1 on failure.

int convertLowerCase ( char *line )

• Convert all alphabetic characters to lower case (do not use the tolower() function in the string library for C or your grade will be zero for this function).
• Returns the number of characters converted from upper case to lower case.

int replaceDigits ( char *line )

• Replace the all numbers/digits (0-9) with blanks.
• Returns the number of digits replaced with blanks.

int replacePunc ( char *line )

• Replace all punctuation and symbols with blanks. A list of all punctuation and symbols appears in the Information section of this assignment.
• Returns the number of punctuation and symbols replaced with blanks.

int reduceSpace ( char *line )

• Reduce all white space (blank characters and tabs to a single blank space).
• Returns the number of white spaces removed.

int trim ( char *line )

• Trim all white space from the beginning and from the end of each line.
• Returns 0 on success and -1 on failure.

Now compile the program findWords.c with your functions using Makefile and run this against the test file called verneTest.txt.

$ make findWords $ cat verneTest.txt | ./findWords

Your output should look like the output in file resultsVerne.txt.

The findWords program has found all the words in the file that are greater than 5 characters in length.

Now run the following pipeline:

$ cat verneTest.txt | ./findWords | sort

The output should look like the output in file sortedResultsVerne.txt.

Your last task is to write a program called wordBag.c that prints out all the unique words produced by the previous pipeline with their counts (number of times that they occur). The output will look like the output in file countsResultsVerne.txt.

Now run the following pipelines:

$ cat verneTest.txt | ./findWords | sort | ./wordBag $ cat verneTest.txt | ./findWords | sort | ./wordBag | sort gk1,1r -gk2,2

Now you have a set of programs and functions that show you how popular specific words are in a text.

Information ASCII Table all punctuation and symbols are indicated in the blue boxes.

http://www.asciitable.com/