[Solved] CSE34 Project 1-Lexical Analysis

The goal of this project is to give you hands-on experience with lexicalanalysis. You will extend the provided lexical analyzer to support moretoken types. The next section lists all new token types that you need toimplement.1. Token TypesModify the lexer to support the following 3 token types:REALNUM = NUM DOT digit digit*BASE08NUM = ((pdigit8 digit8*) + 0) (x) (08)BASE16NUM = ((pdigit16 digit16*) + 0) (x) (16)Wheredigit16 = 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + A + B + C + D + E + Fpdigit16 = 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9 + A + B + C + D + E + Fdigit = 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9pdigit = 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9digit8 = 0 + 1 + 2 + 3 + 4 + 5 + 6 + 7pdigit8 = 1 + 2 + 3 + 4 + 5 + 6 + 7Note that NUM and DOT are already defined in the lexer, but here arethe regular expressions for the sake of completeness:NUM = (pdigit digit*) + 0DOT = .Note that DOT is a single dot character, the quotes are used to avoidambiguity.The list of valid tokens including the existing tokens in the code wouldbe as follows:IFWHILEDOTHENPRINTPLUSMINUSDIVMULTEQUALCOLONCOMMASEMICOLONLBRACRBRACLPARENRPARENNOTEQUALGREATERLESSLTEQGTEQDOTNUMIDREALNUMBASE08NUMBASE16NUMThis list should be used to determine the token if the input matchesmore than one regular expression.2. How-ToFollow these steps:Download the lexer.cc , lexer.h , inputbuf.cc and inputbuf.h filesaccompanying this project description. Note that these files mightbe a little different than the code youve seen in class orelsewhere.Add your code to the files to support the token types listed in theprevious section.Compile your code using GCC compiler on CentOS 6.7 . You will needto use the g++ command to compile your code in a terminal window.Note that you are required to compile and test your code in CentOS6.7 using the GCC compilers. You are free to use any IDE or texteditor on any platform, however, using tools available in CentOS(or tools that you could install on CentOS could save time in thedevelopment/compile/test cycle. See next section for more detailson how to compile using GCC.Test your code to see if it passes the provided test cases. Youwill need to extract the test cases from the zip file and run thetest script test1.sh . More details on this in the next section.Submit your code in canvas before the deadline:3. Compile & Test3.1 Compiling Code with GCCYou should compile your programs with the GCC compilers which areavailable in CentOS 6.7. The GCC is a collection of compilers for manyprogramming languages. There are separate commands for compiling C andC++ programs:Use gcc command to compile C programsUse g++ to compile C++ programsHere is an example of how to compile a simple C++ program:$ g++ test_program.cppIf the compilation is successful, gcc will generate an executable filenamed a.out in the same folder as the program. You can change the outputfile name by specifying the -o switch:$ g++ test_program.cpp -o hello.outTo enable all warning messages of the GCC compiler, use the -Wallswitch:$ g++ -Wall test_program.cpp -o hello.outThe same options can be used with gcc to compile C programs.Compiling projects with multiple filesIf your program is written in multiple source files that should belinked together, you can compile and link all files together with onecommand:$ g++ file1.cpp file2.cpp file3.cppOr you can compile them separately and then link:$ g++ -c file1.cpp$ g++ -c file2.cpp$ g++ -c file3.cpp$ g++ file1.o file2.o file3.oThe files with the .o extension are object files but are notexecutable. They are linked together with the last statement and thefinal executable will be a.out .NOTE: you can replace g++ with gcc in all examples listed above tocompile C programs.3.2 Testing your code with I/O RedirectionYour programs should not explicitly open any file. You can only use thestandard input e.g. std::cin in C++, getchar() , scanf() in C andstandard output e.g. std::cout in C++, putchar() , printf() in C forinput/output.However, this restriction does not limit our ability to feed input tothe program from files nor does it mean that we cannot save the outputof the program in a file. We use a technique called standard IOredirection to achieve this.Suppose we have an executable program a.out , we can run it by issuingthe following command in a terminal (the dollar sign is not part of thecommand):$ ./a.outIf the program expects any input, it waits for it to be typed on thekeyboard and any output generated by the program will be displayed onthe terminal screen.Now to feed input to the program from a file, we can redirect thestandard input to a file:$ ./a.out < input_data.txtNow, the program will not wait for keyboard input, but rather read itsinput from the specified file. We can redirect the output of the programas well:$ ./a.out > output_file.txtIn this way, no output will be shown in the terminal window, but ratherit will be saved to the specified file. Note that programs have accessto another standard interface which is called standard error e.g.std::cerr in C++, fprintf(stderr, ) in C. Any such output is stilldisplayed on the terminal screen. However, it is possible to redirectstandard error to a file as well, but we will not discuss that here.Finally, its possible to mix both into one command:$ ./a.out < input_data.txt > output_file.txtWhich will redirect standard input and standard output to input_data.txtand output_file.txt respectively.Now that we know how to use standard IO redirection, we are ready totest the program with test cases.Test CasesA test case is an input and output specification. For a given inputthere is an expected output. A test case for our purposes is usuallyrepresented by two files:test_name.txttest_name.txt.expectedThe input is given in test_name.txt and the expected output is given intest_name.txt.expected .To test a program against a single test case, first we execute theprogram with the test input data:$ ./a.out < test_name.txt > program_output.txtThe output generated by the program will be stored in program_output.txt .To see if the program generated the expected output, we need to compareprogram_output.txt and test_name.txt.expected . We do that using a generalpurpose tool called diff :$ diff -Bw program_output.txt test_name.txt.expectedThe options -Bw tells diff to ignore whitespace differences between thetwo files. If the files are the same (ignoring the whitespacedifferences), we should see no output from diff , otherwise, diff willproduce a report showing the differences between the two files.We would simply consider the test passed if diff could not find anydifferences, otherwise we consider the test failed.Our grading system uses this method to test your submissions againstmultiple test cases. There is also a test script accompanying thisproject test1.sh which will make your life easier by testing your codeagainst multiple test cases with one command.Here is how to use test1.sh to test your program:Store the provided test cases zip file in the same folder as yourproject source filesOpen a terminal window and navigate to your project folder usingcd commandUnzip the test archive using the unzip command:$ unzip test_cases.zipNOTE: the actual file name is probably different, you shouldreplace test_cases.zip with the correct file name.Store the test1.sh script in your project directory as wellMark the script as executable once you download it:$ chmod +x test1.shCompile your program. The test script assumes your executable iscalled a.outRun the script to test your code:$ ./test1.shThe output of the script should be self explanatory. To test your codeafter each change, you will just perform the last two steps afterwards.4. RequirementsHere are the requirements of this project:You should submit your code on the course submission website, noother submission forms will be accepted.You should use C/C++, no other programming languages are allowed.You should familiarize yourself with the CentOS environment and theGCC compiler. Programming assignments in this course might be verydifferent from what you are used to in other classes.