Tutorial: How to Perform Data Analysis on Linux and UNIX
Data Analysis - Introduction
Objective
Given raw data, the student should be able to sort, provide statistically significant information, and provide clear reports.
Overview
The data set provided comes from an apache web server log. This is real log information; however, the text has been truncated at 112 columns to allow for easy on-screen text manipulation.
Commands
vi
cat
sort
uniq
grep
awk
sed
wc
head
tail
rev
tac
Data Set:
visitor, identd_check, authenticated_user, timestamp, method_request_and_reponse, status, bytes, referrer, user_agent
66.249.69.31 - - [03/Mar/2019:03:37:59 +0000] "GET /images/drawings/fox_g9it7.jpg HTTP/1.1" 304 - "-" "Googlebot
66.249.69.31 - - [03/Mar/2019:03:37:59 +0000] "GET /images/drawings/fox_g9it7.jpg HTTP/1.1" 304 - "-" "Googlebot
122.114.127.156 - - [03/Mar/2019:03:41:54 +0000] "GET /user.php?act=login HTTP/1.1" 302 295 "554fcae493e564ee0dc
122.114.127.156 - - [03/Mar/2019:03:41:54 +0000] "GET /user.php?act=login HTTP/1.1" 302 295 "554fcae493e564ee0dc
122.114.127.156 - - [03/Mar/2019:03:41:56 +0000] "GET / HTTP/1.1" 200 57 "554fcae493e564ee0dc75bdf2ebf94caads|a:
122.114.127.156 - - [03/Mar/2019:03:41:56 +0000] "GET / HTTP/1.1" 200 57 "554fcae493e564ee0dc75bdf2ebf94caads|a:
122.114.127.156 - - [03/Mar/2019:03:41:57 +0000] "GET /user.php?act=login HTTP/1.1" 302 295 "45ea207d7a2b68c4958
122.114.127.156 - - [03/Mar/2019:03:41:57 +0000] "GET /user.php?act=login HTTP/1.1" 302 295 "45ea207d7a2b68c4958
122.114.127.156 - - [03/Mar/2019:03:41:57 +0000] "GET / HTTP/1.1" 200 57 "45ea207d7a2b68c49582d2d22adf953aads|a:
122.114.127.156 - - [03/Mar/2019:03:41:57 +0000] "GET / HTTP/1.1" 200 57 "45ea207d7a2b68c49582d2d22adf953aads|a:
59.56.241.218 - - [03/Mar/2019:03:42:35 +0000] "GET / HTTP/1.1" 200 5087 "-" "Mozilla/5.0 (Windows NT 6.1) Apple
59.56.241.218 - - [03/Mar/2019:03:42:35 +0000] "GET / HTTP/1.1" 200 5087 "-" "Mozilla/5.0 (Windows NT 6.1) Apple
52.53.201.78 - - [03/Mar/2019:03:43:05 +0000] "GET / HTTP/1.1" 200 6049 "-" "Mozilla/5.0 (Macintosh; Intel Mac O
52.53.201.78 - - [03/Mar/2019:03:43:05 +0000] "GET / HTTP/1.1" 200 6049 "-" "Mozilla/5.0 (Macintosh; Intel Mac O
34.235.233.220 - - [03/Mar/2019:03:50:49 +0000] "GET / HTTP/1.1" 200 13894 "-" "Mozilla/5.0 (Windows NT 10.0; Wi
34.235.233.220 - - [03/Mar/2019:03:50:49 +0000] "GET / HTTP/1.1" 200 13894 "-" "Mozilla/5.0 (Windows NT 10.0; Wi
3.89.137.200 - - [03/Mar/2019:03:52:25 +0000] "GET /california/homeschool_resources.html HTTP/1.1" 200 16335 "-"
3.89.137.200 - - [03/Mar/2019:03:52:25 +0000] "GET /california/homeschool_resources.html HTTP/1.1" 200 16335 "-"
148.251.195.14 - - [03/Mar/2019:03:55:23 +0000] "GET /robots.txt HTTP/1.1" 302 292 "-" "Mozilla/5.0 (compatible;
148.251.195.14 - - [03/Mar/2019:03:55:23 +0000] "GET /robots.txt HTTP/1.1" 302 292 "-" "Mozilla/5.0 (compatible;
125.160.139.255 - - [03/Mar/2019:03:55:23 +0000] "GET / HTTP/1.1" 200 184 "-" "Mozilla/5.0 (Windows NT 6.1; WOW6
125.160.139.255 - - [03/Mar/2019:03:55:23 +0000] "GET / HTTP/1.1" 200 184 "-" "Mozilla/5.0 (Windows NT 6.1; WOW6
148.251.195.14 - - [03/Mar/2019:03:55:25 +0000] "GET / HTTP/1.1" 200 57 "-" "Mozilla/5.0 (compatible; MJ12bot/v1
148.251.195.14 - - [03/Mar/2019:03:55:25 +0000] "GET / HTTP/1.1" 200 57 "-" "Mozilla/5.0 (compatible; MJ12bot/v1
148.251.195.14 - - [03/Mar/2019:03:55:26 +0000] "GET /robots.txt HTTP/1.1" 302 296 "-" "Mozilla/5.0 (compatible;
The following questions are "as recorded in this log."
1) Save the data set as a file
vi lab01.log
i ### insert mode
paste the data
press [ESC]
:wq ### write and quit the file
2) How many diffent visitors were recorded to the web server?
3) How many times did each visitor IP Address access websites on this server?
4) How many times was the request valid (200-series reponse)
5) How many times was the request redirected (300-series response)
6) How many times did robots scrape webpages on the server?
7) How many times did hackers try to login to websites on the server?
8) How would you sort just the visitor IP and their browser?
Thank you for dropping by!
Please join me:
/ paulwpoteete
/ paulwpoteete
/ paulwpoteete
/ paulwpoteete
https://plus.google.com/+PaulWPoteete
/ paulwpoteete
#linux, #unix, #tutorial, #administration, #engineering, #beginner, #expert, #advanced, #intermediate, #training
Auf dieser Seite können Sie das Online-Video Tutorial: How to Perform Data Analysis on Linux and UNIX mit der Dauer stunde minuten sekunde in guter Qualität ansehen, das der Benutzer The Expat Professor 07 März 2019 hochgeladen hat, den Link mit Freunden und Bekannten teilen, dieses Video wurde auf Youtube bereits 1,846 Mal angesehen und es wurde von like den Zuschauern gefallen. Viel Spaß beim Betrachtenden Zuschauern gefallen!
