OPS145 Lab 5: Difference between revisions

From Littlesvr Wiki
Jump to navigation Jump to search
No edit summary
No edit summary
Line 1: Line 1:
=REDIRECTION: STANDARD INPUT / STANDARD OUTPUT / STANDARD ERROR=
= !!! THIS PAGE IS NOT READY YET !!! =
<br>
One of the most common tasks a system administrator will do is to modify some configuration. Almost all services on Linux servers are configured using plain text configuration files.
===Main Objectives of this Practice Tutorial===


:* Understand and use the '''cut''', '''tr''', and '''wc''' Linux commands
So far in the course you've been using a graphical text editor. But since Linux servers don't have graphical user interfaces (GUIs): you have to learn to edit text files on the command line.


:* Define the terms '''Standard Input''' (''stdin''), '''Standard Output''' (''stdout''), and '''Standard Error''' (''stderr'')
There are several common applications for command line text file editing. We're going to learn '''vi'''. Even though vi is a frustratingly annoying application to use: it's the one available on any Linux system.


:* Understand and use the '''>''', '''>>''', '''2>''', '''2>>''' symbols with Linux commands
= vi =
As you go through your Linux courses: try to think of vi as a necessary evil. We're going to learn the absolute basics necessary for you to be able to accomplish any task a systems administrator would need it for.


:* Understand the purpose of the the '''/dev/null''' file and the '''Here Document'''
== Modes ==
One of the confusing things about vi is that it can be in one of two modes at any time: command mode, and insert mode.


:* Define the term '''pipeline command''' and explain how a pipeline command functions
In '''insert mode''' it functions more or less the way you'd expect a text editor to function, except depending on your setup: the arrow keys, delete key, and backspace key might do weird things. Otherwise: when you press a letter key: that gets inserted into the text.


:* Define the term '''filter''' and how it relates to pipeline commands
In '''command mode''' the keypresses on your keyboard are interpreted as instructions/commands to vi, and different keys are used for different commands.


:* Use the '''semicolon''' ";" and '''grouping''' "( )" symbols to issue multiple Unix / Linux commands on a single line
For example: in insert mode pressing '''x''' on your keyboard will put the letter x into the text where the cursor is, but in command mode pressing '''x''' on your keyboard will delete the character underneath the cursor.


:* Use the '''backslash''' "\" symbol to spread-out long Unix/Linux commands over multiple lines
Often times this creates such confusion (especially if you don't look at the screen when you're typing) that you have to abandon what you've been trying to do and start over. A good portion of this lab is spent on recovery from mistakes.
<br>


===Tutorial Reference Material===
== Quit without saving ==


{|width="100%" cellspacing="0" cellpadding="10"
* Open one of the text files you donwloaded in lab2 in vi:<syntaxhighlight lang="bash">
vi ~/Downloads/SampleFiles/1984.txt
</syntaxhighlight>


|- valign="top"
Similarly to the '''less''' command: you can move around using the arrows, home/end, and PgUp/PgDown keys. Unlike the less command: there is a cursor. But if you try to type in some text: depending on which keys you use either nothing will happen or something weird will happen.


|colspan="2" style="font-size:16px;font-weight:bold;border-bottom: thin solid black;border-spacing:0px;padding-left:15px;"|Linux Command / Shortcut Reference<br>
* Move the cursor to an empty like, and type in '''hello'''. Whatever happend in vi is probably not what you wanted.
* Press '''ctrl+s'''. This will not save your document. Try '''ctrl+w''', and '''ctrl+q'''. Neither will quit out of vi.
* Try the '''backspace''' and '''delete''' keys. They may or may not work, depending on various complicated factors.


|- valign="top" style="padding-left:15px;"
At this point you probably made some unintended changes to your document, and you don't know how to undo them. The best thing to do when you're in this situation is to quit vi without saving:
|  style="padding-left:15px;" |'''Redirection:'''
* [http://www.linfo.org/standard_input.html Standard Input (stdin)]
* [http://www.linfo.org/standard_output.html Standard Output (stdout)]
* [http://www.linfo.org/standard_error.html Standard Error (stderr)]
* [http://www.linfo.org/pipe.html Pipeline Commands]<br><br>
'''Multiple Commands:'''
* [https://www.javatpoint.com/linux-semicolon Semicolon]<br>
* [https://www.gnu.org/software/bash/manual/html_node/Command-Grouping.html Grouping ( )]<br><br>
|  style="padding-left:15px;"|'''Redirection Filters:'''
* [http://man7.org/linux/man-pages/man1/more.1.html more] , [http://man7.org/linux/man-pages/man1/less.1.html less]<br>
* [http://man7.org/linux/man-pages/man1/head.1.html head] , [http://man7.org/linux/man-pages/man1/tail.1.html tail]
* [http://man7.org/linux/man-pages/man1/sort.1.html sort]<br>
* [http://man7.org/linux/man-pages/man1/uniq.1.html uniq]<br>
* [http://linuxcommand.org/lc3_man_pages/grep1.html grep]<br>
* [http://man7.org/linux/man-pages/man1/cut.1.html cut]<br>
* [http://linuxcommand.org/lc3_man_pages/tr1.html tr]<br>
* [http://man7.org/linux/man-pages/man1/wc.1.html wc]<br>
* [http://man7.org/linux/man-pages/man1/tee.1.html tee]<br>
|}


= KEY CONCEPTS =
* Press the '''Escape''' key, then colon (''':'''), then '''q''', then exclamation mark ('''!'''), then '''Enter'''. From now on in the instructions I'll use a shorthand notation like this: '''Esc'''+''':q!'''


===Additional File Manipulation Commands===
This will quit vi without saving any changes you made to the document.


Before proceeding, let's look at some additional commands used to manipulate content of text files.
* You will also want to do this if you open vi with no filename, or the wrong filename. Try it:<syntaxhighlight lang="bash">
vi ~/Downloads/SampleFiles/1985.txt
</syntaxhighlight>
* You could put in some text, and save it: but it would be saved in the wrong file. So quit without saving using '''Esc'''+''':q!'''
{{Admon/important|Never use ctrl+z|If you've taken this course with someone else: you may have learned about ctrl+z. '''You are not qualified to understand what that does, and using it will cause you more trouble than good.''' Just forget that that key combination exists!}}


Refer to the table below regarding these text file manipulation commands:
== Switching modes ==
<br><br>
[[Image:manipulation-commands.png|left|700px|]]
<br><br><br><br><br><br><br><br><br>


===Redirection (Standard Input, Standard Output, Standard Error)===
* Open 1984.txt again, use an absolute path for practice:<syntaxhighlight lang="bash">
vi /home/yourusername/Downloads/SampleFiles/1984.txt
</syntaxhighlight>
* Move your cursor to the fourth line using the arrow keys.
* Press '''i''' - this will switch vi into '''insert mode'''. Depending on the version of vi: it may or may not tell you at the bottom that it's now in insert mode.
* Type in '''This is a new sentence!''' - this will insert that text at the cursor position.
* Press '''Escape'''. This will take vi into '''command mode'''.
* Save your changes using the ''':w''' command.
* Quit vi using the ''':q''' command.
* Use '''less''' to read 1984.txt and confirm that your changes have been saved.


<i>'''Redirection''' can be defined as changing the way from where commands read input to where commands sends output.<br>You can redirect input and output of a command.</i>
== Append ==
The '''i''' command will switch vi into insert mode, and will keep the cursor where it is. Sometimes you need to insert some text at the end of a line, and because of your configuration the arrow keys won't work in insert mode. Try it:


Reference: https://www.javatpoint.com/linux-input-output-redirection
* Open 1984.txt again using vi
* Move your cursor to the end of the line you typed in (over the exclamation mark), and switch to insert mode using '''i'''.


[[Image:stdin-symbol.png|thumb|right|250px|The '''standard input''' ('''stdin''') symbol that describes where a Unix/Linux command receives '''input''']]
Note that the cursor is over the exclamation mark, but you will need to add text after it.
'''Standard input''' ('''stdin''') is a term which describes from where a command receives '''input'''.<br>
This would apply only to Unix/Linux commands that accept stdin input<br>(like ''cat'', ''more'', ''less'', ''sort'', ''grep'', ''head'', ''tail'', ''tr'', ''cut'', ''wc'', etc.).<br>


''Examples:''
* Try to move the cursor to the right using the arrow key. If your configuration is the same as my defaults: vi will do something stupid.
* Quit vi without saving your changes.


<span style="font-family:courier;font-weight:bold">tr 'a-z' 'A-Z' < words.txt<br>cat < abc.txt<br>sort < xyz.txt</span>
There's another command which will switch vi into insert mode, but unlike '''i''': it will also move the cursor one character forward. That command is '''a''' (for append). Try it:


* Open 1984.txt again using vi
* Move your cursor to the end of the line you typed in (over the exclamation mark), and switch to insert mode using '''a'''.
* Type in " '''This is a second sentence. And a third.'''" You should end up with this:


<table align="right"><tr><td>[[Image:stdout-symbol-1.png|thumb|right|250px|The '''standard out''' ('''stdout''') symbol with one greater than sign '''overwrites''' existing file content with command output]]</td><td>[[Image:stdout-symbol-2.png|thumb|right|250px|The '''standard output''' ('''stdout''') symbol  with two greater than signs '''add''' command's output to '''bottom''' of existing file's contents.]]</td></tr></table>
[[File:ViInsertedTwoSentences.png|center]]
'''Standard output''' ('''stdout''') describes where a command sends its '''output'''.<br>In the examples below, output from a command is sent to the '''monitor''', unless it is sent to a '''text file'''.


* Save your changes, exit vi, and confirm that your changes were saved correctly.


''Examples:''
== Delete one character ==
Sometimes the delete and backspace keys work properly, but half the time they dont. A relatively easy way to delete some text is to use the '''x''' command, in command mode.


<span style="font-family:courier;font-weight:bold">ls -l<br>ls -l > detailed-listing.txt<br>ls /bin >> output.txt</span>
*Open 1984.txt again using vi
* Move your cursor to the space between the second and third sentence you added.
* Still in command mode: press '''x'''. This will delete the space.
* Press '''x''' several more times to delete the rest of the third sentence.


If you make a mistake (e.g. delete too much): quit vi without saving, and try again.


<table align="right"><tr><td>[[Image:stderr-symbol-1.png|thumb|right|250px|The '''standard error''' ('''sterr''') symbol with one greater than sign '''overwrites''' existing file content with command's '''error message'''.]]</td><td>[[Image:stderr-symbol-2.png|thumb|right|250px|The '''standard error''' ('''stderr''') symbol  with two greater than signs '''add''' command's error message to '''bottom''' of existing file's contents.]]</td></tr></table>
You should end up with this:
'''Standard Error''' ('''stderr''') describes where a command sends it's error messages. In the examples below we issue the pwd in capitals on purpose to generate an error message, which can be redirected to a '''text file'''.
[[File:ViThirdSentenceDeleted.png|center]]


* Save your changes, exit vi, and confirm that your changes were saved correctly.


''Examples:''
== Search ==
If the file you're editing is more than a couple of pages long: you'll probably want to use the search function. It's relatively painless, just remember that it does a case-sensitive search.


<span style="font-family:courier;font-weight:bold">PWD<br>PWD 2> error-message.txt<br>PWD 2 >> error-messages.txt<br>PWD 2> /dev/null</span>
* Open 1984.txt again using vi
<br><br>


====The /dev/null File====
You're going to have to find a note at the end of the book. That note contains the string '''Australia'''.


* Still in command mode: press the forward slash ('''/''') followed by '''Australia''', then '''Enter'''
* The first place the string "Australia" shows up is not what you're looking for. Find the next occurence using just slash and enter.
* You should end up with your cursor on one of the last lines in the file:


The '''/dev/null''' file (sometimes called the '''bit bucket''' or  '''black hole''') is a special system file<br>that '''discard''' all  data written into it. This is useful to discard unwanted command output.
[[File:ViAustraliaFound.png|center]]


* Leave it there for the next part you need to do.


''Examples:''
== Delete one line ==
When you need to delete a whole line (rather than part of one): you can use the '''dd''' vi command. I don't know what that stands for. Delete-da-line or something.


<span style="font-family:courier;font-weight:bold;">LS 2> /dev/null<br>ls > /dev/null<br>find / -name "tempfile" 2> /dev/null</span>
* With your cursor anywhere on the line with Project Gutenberg: press '''dd''' (press '''d''' twice). That line will be deleted.
<br><br>
* Repeat the command a few times so that only one empty line is left after THE END:


====The Here Document====
[[File:ViLastLinesDeleted.png|center]]


* Save your changes, exit vi, and confirm that your changes were saved correctly. In the '''less''' command you can press the '''End''' key to go to the end of the file.


[[Image:Here Document.png|thumb|right|175px|The '''Here Document''' allows a user to redirect stdin from <u>within</u> the command itself.]]
== Undo ==
In Linux, the '''Here Document''' allows a user to redirect stdin from within the command itself.
If you've been working on a file for some time, and then accidentally press the wrong key (for example because the arrow keys don't work): you can use the undo feature to avoid redoing all your edits. It only undoes the one last operation performed but that can still be helpful. Try it out:


''Example:''
* Open 1984.txt again using vi.
* Switch to insert mode.
* Press one of the arrow keys. If your vi is in the same default configuration as mine: instead of the cursor moving it will insert a new line with a character on it.
* Switch back to command mode.
* Press '''u''' - it should remove the extra line with the extra character.
* The second time you press '''u''' will redo whatever was undone.


<span style="font-family:courier;font-weight:bold">cat <<+<br>Line 1<br>Line 2<br>Line 3<br>+<br><br>
= More vi practice =


===Pipeline Commands===
* Download this file: [http://ops345.ca/ops145/ViExercise.tar.xz ViExercise.tar.xz] and extract it in the Downloads directory.


[[Image:pipe-diagram-1.png|thumb|right|450px|A '''pipeline command''' sends a command's '''standard output''' directly to '''standard input''' of other command(s) without having to create temporary files.]]
The tarball contains a directory which you don't have read permissions to. This is to encourage you to use vi to complete this section of the lab instead of using a graphical text editor. Of course if you're stubborn: you have learned by now how to change permissions on files which you own.
'''Pipeline Command:''' Having commands send their '''standard  output''' <u>directly</u> to '''standard input''' of other  commands WITHOUT having to use '''temporary''' files.
 
Pipes that are used in a '''pipeline command''' are represented by the '''pipe''' "|" symbol.<br>
 
A few simple commands can be '''combined''' to form a more <u>powerful</u> command line.<br>
 
 
Commands to the '''right''' of the pipe symbol are referred to as '''filters'''. They are referred to as ''filters'' since those commands are used to '''modify''' the stdout of the <u>previous command</u>. Many commands can be "piped" together, but these commands (filters) must be chained in a specific order, depending on what you wish to accomplish
 
 
''Examples:''<br>
<span style="font-family:courier;font-weight:bold">ls -al | more<br>ls | sort -r<br>ls | sort | more<br>ls -l | cut -d" " -f2 | tr 'a-z' 'A-z"<br>ls | grep Linux | head -5<br>head -7 filename | tail -2</span>
 
====The tee Command====
 
 
[[Image:tee-diagram-1.png|thumb|right|250px|The '''tee''' utility can be used to '''split''' the flow of information. For example to save in a file as well as display on a screen. <br>(Image licensed under [https://creativecommons.org/licenses/by-sa/3.0/ cc])]]
The '''tee''' utility can be used to <u>split</u> the flow of '''standard output'''<br>between a '''text file''' and the '''terminal screen'''.<br><br>
The '''tee''' option '''-a''' can be used to add content to the '''bottom''' of an existing file<br>as opposed to ''overwriting'' the file's previous contents.
 
The reason for the name "'''tee'''" is that the splitting of the flow of information resembles a capital T.
 
 
''Examples:''
<span style="font-family:courier;font-weight:bold">ls | tee unsorted.txt | sort<br>ls | grep Linux | tee matched.txt | more<br>ls | head -5 | tee -a listing.txt</span>
<br><br>
 
===Multiple Commands Using Semicolon, Grouping, and Backquotes===
Besides piping, there are other ways that multiple commands may be placed in one line:<br>commands may be separated by '''semi-colons'''.<br>
 
 
''Example:''
 
<span style="font-family:courier;font-weight:bold">sleep 5; ls</span> 
 
 
Multiple commands can also be '''grouped''' by using parentheses.
 
 
''Example:''
 
<span style="font-family:courier;font-weight:bold">(echo "Who is on:"; w) > whoson</span><br>('''''Note:''' <u>all</u> command output is sent to a file'')
 
 
Commands may also be '''spread-out over multiple lines''', making it easier (for humans) to interpret a long command.<br><br>
The '''\''' symbol “''quotes-out''” the meaning of the '''ENTER''' key as <u>text</u><br> (i.e. ''new-line'' as instead of ''running'' the command).
 
 
''Example:''
 
<span style="font-family:courier;font-weight:bold">echo "This will be split over multiple \<br>lines.  Note that the shell will realize \<br>that a pipe requires another command, so \<br>it will automatically go to the next line" |tr '[a-z]' '[A-Z]'</span>
<br><br>
 
=INVESTIGATION 1: BASICS OF REDIRECTION=
 
<span style="color:red;">'''ATTENTION''': This online tutorial will be required to be completed by '''Friday in week 6 by midnight''' to obtain a grade of '''2%''' towards this course</span><br><br>
 
In this investigation, you will learn how to redirect '''standard input''', '''standard output''' and '''standard error''' when issuing Unix / Linux commands.
 
 
'''Perform the Following Steps:'''
 
# '''Login''' to your matrix account and issue a command to '''confirm''' you are located in your '''home''' directory.<br><br>
# Issue the following Linux command to create the following directory: <span style="color:blue;font-weight:bold;font-family:courier;">mkdir ~/redirect</span><br><br>
# Change to the '''~/redirect''' directory and confirm that you changed to that directory.<br><br>
# Use a text editor to create a file in your current directory called '''data.txt''' and enter the following text displayed below:<br><br><span style="font-family:courier;font-weight:bold">This is line 1<br>This is line 2<br>This is line 3</span><br><br>
# '''Save''' editing changes and '''exit''' the text editor.<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' < data.txt</span><br><br>What does this command do?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' < data.txt > output.txt</span><br><br>What does this command do? What are the contents of the file ''output.txt''?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' > output.txt < data.txt</span><br><br>What does this command do? Is there any difference in terms of this command and the previous command issued?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tr 'a-z' 'A-Z' >> output.txt < data.txt</span><br><br>What happens to the content of the '''output.txt''' file? Why?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tail -2 < data.txt > output.txt</span><br><br>What does this command do? Check the contents of the '''output.txt''' file to confirm.<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">tail -2 > output2.txt < data.txt </span><br><br>Why does this command render the same results as the previous command?<br>Try explaining how the command works in terms of '''stdin''' and then '''stdout'''.<br><br>
# Issue the following Linux command to create a file: <span style="color:blue;font-weight:bold;font-family:courier;">cat > output3.txt </span><br><br>
# Enter the follow text displayed below:<br><br><span style="font-family:courier;font-weight:bold;">This is the file output3.txt</span><br><br>
# Press <span style="color:blue;font-weight:bold;font-family:courier;">ctrl-d</span> to exit the command.<br><br>
# Issue the '''cat''' command to view the contents of the file: '''output3.txt'''<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cp ~jason.carman/uli101/cars .</span><br><br>
# Issue the '''cat''' command to view the contents of the '''cars''' file.<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -c1-10 cars</span><br><br>What did this command do?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f5 cars > field5.txt</span><br><br>What did this command do?<br>Check the contents in the file '''field5.txt''' to see what happened.<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f1-3 cars > field123.txt</span><br><br>What did this command do? (check file contents)<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cut -f1,5 cars > field15.txt</span><br><br>What did this command do? (check file contents)<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">wc cars > count.txt</span><br><br>What information does the '''count.txt''' file contain?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">wc -l cars > count1.txt</span><br><br>What information does the '''count1.txt''' file contain?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">wc -w cars > count2.txt</span><br><br>What information does the '''count2.txt''' file contain?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">ls -l > listing.txt</span><br><br>What information does the '''listing.txt''' file contain?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">pwd > listing.txt</span><br><br>What happenned to the original contents of the file called '''listing.txt'''? Why?<br><br>
# Issue the following Linux command (use 2 greater-than signs): <span style="color:blue;font-weight:bold;font-family:courier;">date >> listing.txt</span><br><br>What information does the '''listing.txt''' file contain? Why?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars > combined.txt</span><br><br>What information does the '''combined.txt''' file contain? Why?<br><br>'''NOTE''': The '''cat''' command stands for "'''concatenate'''" which means to '''combine''' contents of multiple files into a single file.<br>This is why the command is called "''cat''".<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars murray 2> result.txt</span><br><br>What is displayed on the monitor? What information does the '''result.txt''' file contain? Why?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars murray > myoutput.txt 2> /dev/null</span><br><br>What is displayed on the monitor? What happened to the error message?<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">cat listing.txt cars murray > myoutput.txt 2> result.txt</span><br><br>What is displayed on the monitor? what do those files contain? Why?<br><br>The '''Here Document''' allows you to redirect stdin from with the Linux command itself. Let's get some practice using the Here Document.<br><br>
# Issue the following Linux command:<br><span style="color:blue;font-weight:bold;font-family:courier;">cat <<+<br>line 1<br>line 2<br>line 3<br>+</span><br><br>What do you notice?<br><br>
# Issue the following Linux command:<br><span style="color:blue;font-weight:bold;font-family:courier;">grep 2 <<+<br>line 1<br>line 2<br>line 3<br>+</span><br><br>What do you notice? How does this differ from the previous command? Why?<br><br>
# Issue the following Linux command:<br><span style="color:blue;font-weight:bold;font-family:courier;">grep 2 > line2.txt <<+<br>line 1<br>line 2<br>line 3<br>+</span><br><br>What do you notice? What is contained in the file '''line2.txt'''? Why?<br><br>'''NOTE:''' You will now run a shell script to confirm that you properly issued Linux commands using redirection.<br><br>
# Issue the following Linux command to run a checking script:<br><span style="color:blue;font-weight:bold;font-family:courier;">~uli101/week5-check-1</span><br><br>
# If you encounter errors, make corrections and '''re-run''' the checking script until you receive a congratulations message, then you can proceed.<br><br>
# Issue the '''ls''' command to see all of the '''temporary files''' that were created as a result of redirection.<br><br>The problem with using these redirection symbols is that you create '''temporary text files''' that take up '''space''' on your file system.<br><br>
# Issue a Linux command (using '''Filename Expansion''') to '''remove''' those temporary text files in the current directory.<br><br>
# Issue the following Linux command to check that you removed ALL of those temporary text files:<br><span style="color:blue;font-weight:bold;font-family:courier;">~uli101/week5-check-2</span><br><br>
# If you encounter errors, make corrections and '''re-run''' the checking script until you receive a congratulations message, then you can proceed.<br><br>
 
:In the next investigation, you will be learning how to issue '''pipeline Linux commands''' which can<br>accomplish tasks <u>without</u> creating temporary files.<br><br>
 
=INVESTIGATION 2: REDIRECTION USING PIPELINE COMMANDS =
 
In this investigation, you will learn to issue '''pipeline commands''' to to accomplish tasks <u>without</u> having to generate temporary files.
 
 
 
'''Perform the Following Steps:'''
 
# Confirm that you are still located in the '''~/redirect''' directory.<br><br>The '''problem''' with creating temporary files, is that they take up space on your server,<br>and should be removed. You actually did that in the previous investigation.<br><br>You will be issuing a '''pipeline command''' which will use the pipe symbol "|"<br>that will send the stdout from a command as stdin into another command<br><u>without</u> having to create temporary files.<br><br>
# Issue the follow Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin | more</span><br><br>What happened? Press '''q''' to exit display.<br><br>
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin | who</span><br><br>What happened? Although this pipeline command provides output,<br>it '''does <u>not</u> work''' properly as a pipeline command since the '''who''' command is<br>'''NOT''' designed to accept standard input.[[Image:pipe-diagram-1.png|thumb|right|350px|]]<br><br>'''NOTE:''' When issuing pipeline commands, commands to the right of the pipe symbol must be designed to <u>accept</u> '''standard input'''. Since the ''who'' command does not, you did NOT see the contents of the '''/bin''' directory but only information relating to the ''who'' command. Therefore, the '''order''' of which you build your pipeline command and the '''type of command''' that is used as a ''filter'' is extremely important!<br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? > listing.txt</span><br><br>
# Issue the following Linux command: <span style="color:blue;font-weight:bold;font-family:courier;">sort  listing.txt</span><br><br>
# Issue the following Linux command to remove the listing file: <span style="color:blue;font-weight:bold;font-family:courier;">rm listing.txt</span><br><br>
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? | sort </span><br><br>You should notice that the output from this pipeline command is the same output<br>from the command you issued in '''step #5'''.<br><br>
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? | sort | more</span><br><br>What is difference with this pipeline command as opposed to the <u>previous</u> pipeline command? Press '''q''' to exit display.<br><br>
# Issue the '''ls''' command.<br><br>You should notice that '''no files have been created'''.<br>Let's get practice issuing more pipeline commands using commands<br>(previously learned or new) to be used as '''filters'''.<br><br>
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/?? | sort | head -5</span><br><br>What did you notice?<br><br>
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/???? | sort | grep r | tail -2</span><br><br>What did you notice? Could you predict the output prior to issuing this pipeline command?<br><br>
# Issue the following Linux '''pipeline command''': <span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/???? | sort | grep r | cut -c1-6</span><br><br>Try to explain step-by-step each process in the pipeline command (including ''filters'')<br>to explain the final output from this pipeine command.<br><br>
# Confirm that you are still located in the '''~/redirect''' directory.<br><br>
# Issue the following Linux '''pipeline command''':<br><span style="color:blue;font-weight:bold;font-family:courier;">ls /bin/???? | tee unsort.txt | sort | tee sort.txt | grep r | tee match.txt | head </span><br><br>
# Issue the '''ls''' command to view the contents of this redirectory.<br><br>What did you notice?<br><br>
# View the <u>contents</u> of the '''text files''' that were created to see how the '''tee''' command<br>was used in the previous pipeline command.<br><br>What was the purpose of using the '''tee''' command for this pipeline command?<br><br>You will now run a shell script to confirm that you properly issued that Linux pipeline command<br>using the '''tee''' command and redirection.<br><br>
# Issue the following Linux command to run a checking script:<br><span style="color:blue;font-weight:bold;font-family:courier;">~uli101/week5-check-3</span><br><br>If you encounter errors, make corrections and '''re-run''' the checking script until you receive<br>a congratulations message, then you can proceed.<br><br>
# Change to <u>your</u> '''home''' directory.<br><br>
# Remove the '''~/redirect''' directory and its contents.<br><br>
 
:In the next investigation, you will learn various techniques to issue '''multiple Linux commands'''<br>on the same line, or issue a '''single Linux command over multiple lines'''.
<br><br>
 
=INVESTIGATION 3: ISSUING MULTIPLE UNIX/LINUX COMMANDS=
 
In this investigation, you will learn how to issue multiple Unix / Linux commands in a single line or over multiple lines.
 
 
'''Perform the Following Steps:'''
 
# Confirm you are located in your '''home''' directory in your Matrix account.<br><br>
# Issue the following Linux commands (using the ''semicolon'' character "''';'''" to separate <u>each</u> Linux command):<br><span style="color:blue;font-weight:bold;font-family:courier;">cal;pwd;date</span><br><br>Note the  output as well as the <u>order</u> of what each Linux command results.<br><br>
# Issue the following Linux commands: <span style="color:blue;font-weight:bold;font-family:courier;">(cal;pwd;date)</span><br><br>Was there any difference in the output of this command as opposed to the previous command?<br><br>Let's see how grouping affects working with redirection.<br><br>
# Issue the following Linux commands: <span style="color:blue;font-weight:bold;font-family:courier;">cal;pwd;date > output.txt</span><br><br>What happened? Where is the output for the '''date''' command?<br>Why isn't the output for the '''cal''' and '''pwd''' commands are NOT contained in that file?<br><br>
# Issue a Linux command to view the contents of the file called '''output.txt'''<br><br>What do you notice?<br><br>Let's use '''grouping''' to make modification to the previous command<br><br>
# Issue the following Linux commands: <span style="color:blue;font-weight:bold;font-family:courier;">(cal;pwd;date) > output.txt</span><br><br>What did you notice?<br><br>
# Issue a Linux command to view the contents of the file called '''output.txt'''<br><br>What does ''grouping'' do when issuing multiple Linux commands (separated by a semi-colon ";") that uses redirection?<br><br>
# Issue the following Linux pipeline command (using \ at the end of most lines):<br><span style="color:blue;font-family:courier;font-weight:bold">echo "This will be split over multiple \<br>lines.  Note that the shell will realize \<br>that a pipe requires another command, so \<br>it will automatically go to the next line" |tr '[a-z]' '[A-Z]'</span><br><br>Did the command work? What is the purpose of issuing a Linux command in this way?<br><br>
# Complete the Review Questions sections to get additional practice.
<br><br>
 
= LINUX PRACTICE QUESTIONS =
 
The purpose of this section is to obtain '''extra practice''' to help with '''quizzes''', your '''midterm''', and your '''final exam'''.
 
Here is a link to the MS Word Document of ALL of the questions displayed below but with extra room to answer on the document to
simulate a quiz:
 
https://wiki.cdot.senecacollege.ca/uli101/files/uli101_week5_practice.docx
 
Your instructor may take-up these questions during class. It is up to the student to attend classes in order to obtain the answers to the following questions. Your instructor will NOT provide these answers in any other form (eg. e-mail, etc).
 
When answering Linux command questions, refer to the following Inverted Tree Diagram. The linux directory is contained in your home directory. Assume that you just logged into your Matrix account. Directories are <u>underlined</u>.
 
[[Image:week5-dir.png|thumb|left|300px|]]
 
<br><br><br><br><br><br><br><br><br>
 
 
'''Review Questions:'''
 
# Write a single Linux command to provide a detailed listing of all files in the '''/etc''' directory, sending the output to a file called listing.txt in the “'''projects'''” directory (append output to existing file and use a relative pathname)
# Write a single Linux command to redirect the stderr from the command:<br>'''cat a.txt b.txt c.txt''' to a file called '''error.txt''' contained in the “'''assignments'''” directory. (overwrite previous file’s contents and use only relative pathnames)
# Write a single Linux command: '''cat ~/a.txt ~/b.txt ~/c.txt''' and redirect stdout to a file called “good.txt” to the “tests” directory and stderr to a file called “'''bad.txt'''” to the “'''tests'''” directory. (overwrite previous contents for both files and use only relative-to-home pathnames).
# Write a single Linux command to redirect the stdout from the command:<br>'''cat a.txt b.txt c.txt''' to a file called wrong.txt contained in the “'''projects'''” directory and throw-out any standard error messages so they don’t appear on the screen (append output to existing file and use only relative pathnames).<br><br>
# Write a single Linux '''pipeline command''' to display a detailed listing of the '''projects''' directory but pause one screen at a time to view and navigate through all of the directory contents. Use a relative-to-home pathname.
# Write a single Linux '''pipeline command''' to display the sorted contents (in reverse alphabetical order) of the “'''linux'''” directory. Use a relative pathname.
# Assume that the text file called “'''.answers.txt'''” contains 10 lines. Write a single Linux pipeline command to only displays lines 5 through 8 for this file. Use only relative pathnames.
# Write a single Linux '''pipeline command''' to only display the contents of the “'''assignments'''” directory whose filenames match the pattern “'''murray'''” (both upper or lowercase). Use an absolute pathname.
# Write a single Linux '''pipeline command''' to display the number of characters contained in the file called “'''.answers.txt'''”. Use a relative-to-home pathname.
# Write a single Linux '''pipeline command''' to display the number of lines contained in the file called “'''questions.txt'''”. Use a relative pathname.
# Write a single Linux '''pipeline command''' to display only the first 10 characters of each filename contained in your current directory. Also, there is will be a lot of output, so also pause at each screenful so you can navigate throughout the display contents. Use a relative pathname.
# Create a '''table''' listing each Linux command, useful options that were mentioned in this tutorial for the following Linux commands: '''cut''' , '''tr''' , '''wc''' , and '''tee'''.
 
 
 
_________________________________________________________________________________
 
Author:  Murray Saul
 
License: LGPL version 3
Link:    https://www.gnu.org/licenses/lgpl.html
 
_________________________________________________________________________________


* Open the file ~/Downloads/ViExercise/Practice.txt using vi
* The instructions for what you need to do are in the file. Try not to make unnecessary changes, since the script checking your work isn't very intelligent.
* Save your changes, and quit vi. (You can save and exit at the same time with the ''':wq''' command).


= First half of the course: review =
In class with me: we will use any lecture time remaining to review whatever parts you like from the first half of the course.


=Submit evidence of your work=
After you finish the lab: run the following commands to submit your work:<syntaxhighlight lang="bash">
cd ~
wget http://ops345.ca/check/ops145-lab5-check.sh
chmod 700 ops145-lab5-check.sh
./ops145-lab5-check.sh
</syntaxhighlight>If it says "Your lab5 has been submitted": make a screenshot, and you're done. If it gives you any warnings or errors: you have to fix them and try the ./ops145-lab5-check.sh command again.


[[Category:OPS145]]
[[Category:OPS145]]

Revision as of 23:40, 10 February 2024

!!! THIS PAGE IS NOT READY YET !!!

One of the most common tasks a system administrator will do is to modify some configuration. Almost all services on Linux servers are configured using plain text configuration files.

So far in the course you've been using a graphical text editor. But since Linux servers don't have graphical user interfaces (GUIs): you have to learn to edit text files on the command line.

There are several common applications for command line text file editing. We're going to learn vi. Even though vi is a frustratingly annoying application to use: it's the one available on any Linux system.

vi

As you go through your Linux courses: try to think of vi as a necessary evil. We're going to learn the absolute basics necessary for you to be able to accomplish any task a systems administrator would need it for.

Modes

One of the confusing things about vi is that it can be in one of two modes at any time: command mode, and insert mode.

In insert mode it functions more or less the way you'd expect a text editor to function, except depending on your setup: the arrow keys, delete key, and backspace key might do weird things. Otherwise: when you press a letter key: that gets inserted into the text.

In command mode the keypresses on your keyboard are interpreted as instructions/commands to vi, and different keys are used for different commands.

For example: in insert mode pressing x on your keyboard will put the letter x into the text where the cursor is, but in command mode pressing x on your keyboard will delete the character underneath the cursor.

Often times this creates such confusion (especially if you don't look at the screen when you're typing) that you have to abandon what you've been trying to do and start over. A good portion of this lab is spent on recovery from mistakes.

Quit without saving

  • Open one of the text files you donwloaded in lab2 in vi:
    vi ~/Downloads/SampleFiles/1984.txt
    

Similarly to the less command: you can move around using the arrows, home/end, and PgUp/PgDown keys. Unlike the less command: there is a cursor. But if you try to type in some text: depending on which keys you use either nothing will happen or something weird will happen.

  • Move the cursor to an empty like, and type in hello. Whatever happend in vi is probably not what you wanted.
  • Press ctrl+s. This will not save your document. Try ctrl+w, and ctrl+q. Neither will quit out of vi.
  • Try the backspace and delete keys. They may or may not work, depending on various complicated factors.

At this point you probably made some unintended changes to your document, and you don't know how to undo them. The best thing to do when you're in this situation is to quit vi without saving:

  • Press the Escape key, then colon (:), then q, then exclamation mark (!), then Enter. From now on in the instructions I'll use a shorthand notation like this: Esc+:q!

This will quit vi without saving any changes you made to the document.

  • You will also want to do this if you open vi with no filename, or the wrong filename. Try it:
    vi ~/Downloads/SampleFiles/1985.txt
    
  • You could put in some text, and save it: but it would be saved in the wrong file. So quit without saving using Esc+:q!
Important.png
Never use ctrl+z
If you've taken this course with someone else: you may have learned about ctrl+z. You are not qualified to understand what that does, and using it will cause you more trouble than good. Just forget that that key combination exists!

Switching modes

  • Open 1984.txt again, use an absolute path for practice:
    vi /home/yourusername/Downloads/SampleFiles/1984.txt
    
  • Move your cursor to the fourth line using the arrow keys.
  • Press i - this will switch vi into insert mode. Depending on the version of vi: it may or may not tell you at the bottom that it's now in insert mode.
  • Type in This is a new sentence! - this will insert that text at the cursor position.
  • Press Escape. This will take vi into command mode.
  • Save your changes using the :w command.
  • Quit vi using the :q command.
  • Use less to read 1984.txt and confirm that your changes have been saved.

Append

The i command will switch vi into insert mode, and will keep the cursor where it is. Sometimes you need to insert some text at the end of a line, and because of your configuration the arrow keys won't work in insert mode. Try it:

  • Open 1984.txt again using vi
  • Move your cursor to the end of the line you typed in (over the exclamation mark), and switch to insert mode using i.

Note that the cursor is over the exclamation mark, but you will need to add text after it.

  • Try to move the cursor to the right using the arrow key. If your configuration is the same as my defaults: vi will do something stupid.
  • Quit vi without saving your changes.

There's another command which will switch vi into insert mode, but unlike i: it will also move the cursor one character forward. That command is a (for append). Try it:

  • Open 1984.txt again using vi
  • Move your cursor to the end of the line you typed in (over the exclamation mark), and switch to insert mode using a.
  • Type in " This is a second sentence. And a third." You should end up with this:
ViInsertedTwoSentences.png
  • Save your changes, exit vi, and confirm that your changes were saved correctly.

Delete one character

Sometimes the delete and backspace keys work properly, but half the time they dont. A relatively easy way to delete some text is to use the x command, in command mode.

  • Open 1984.txt again using vi
  • Move your cursor to the space between the second and third sentence you added.
  • Still in command mode: press x. This will delete the space.
  • Press x several more times to delete the rest of the third sentence.

If you make a mistake (e.g. delete too much): quit vi without saving, and try again.

You should end up with this:

ViThirdSentenceDeleted.png
  • Save your changes, exit vi, and confirm that your changes were saved correctly.

Search

If the file you're editing is more than a couple of pages long: you'll probably want to use the search function. It's relatively painless, just remember that it does a case-sensitive search.

  • Open 1984.txt again using vi

You're going to have to find a note at the end of the book. That note contains the string Australia.

  • Still in command mode: press the forward slash (/) followed by Australia, then Enter
  • The first place the string "Australia" shows up is not what you're looking for. Find the next occurence using just slash and enter.
  • You should end up with your cursor on one of the last lines in the file:
ViAustraliaFound.png
  • Leave it there for the next part you need to do.

Delete one line

When you need to delete a whole line (rather than part of one): you can use the dd vi command. I don't know what that stands for. Delete-da-line or something.

  • With your cursor anywhere on the line with Project Gutenberg: press dd (press d twice). That line will be deleted.
  • Repeat the command a few times so that only one empty line is left after THE END:
ViLastLinesDeleted.png
  • Save your changes, exit vi, and confirm that your changes were saved correctly. In the less command you can press the End key to go to the end of the file.

Undo

If you've been working on a file for some time, and then accidentally press the wrong key (for example because the arrow keys don't work): you can use the undo feature to avoid redoing all your edits. It only undoes the one last operation performed but that can still be helpful. Try it out:

  • Open 1984.txt again using vi.
  • Switch to insert mode.
  • Press one of the arrow keys. If your vi is in the same default configuration as mine: instead of the cursor moving it will insert a new line with a character on it.
  • Switch back to command mode.
  • Press u - it should remove the extra line with the extra character.
  • The second time you press u will redo whatever was undone.

More vi practice

The tarball contains a directory which you don't have read permissions to. This is to encourage you to use vi to complete this section of the lab instead of using a graphical text editor. Of course if you're stubborn: you have learned by now how to change permissions on files which you own.

  • Open the file ~/Downloads/ViExercise/Practice.txt using vi
  • The instructions for what you need to do are in the file. Try not to make unnecessary changes, since the script checking your work isn't very intelligent.
  • Save your changes, and quit vi. (You can save and exit at the same time with the :wq command).

First half of the course: review

In class with me: we will use any lecture time remaining to review whatever parts you like from the first half of the course.

Submit evidence of your work

After you finish the lab: run the following commands to submit your work:

cd ~
wget http://ops345.ca/check/ops145-lab5-check.sh
chmod 700 ops145-lab5-check.sh
./ops145-lab5-check.sh

If it says "Your lab5 has been submitted": make a screenshot, and you're done. If it gives you any warnings or errors: you have to fix them and try the ./ops145-lab5-check.sh command again.