Tutorial: How to Perform Data Analysis on Linux and UNIX
Data Analysis - Introduction
Objective
Given raw data, the student should be able to sort, provide statistically significant information, and provide clear reports.
Overview
The data set provided comes from an apache web server log. This is real log information; however, the text has been truncated at 112 columns to allow for easy on-screen text manipulation.
Commands
vi
cat
sort
uniq
grep
awk
sed
wc
head
tail
rev
tac
Data Set:
visitor, identd_check, authenticated_user, timestamp, method_request_and_reponse, status, bytes, referrer, user_agent
66.249.69.31 - - [03/Mar/2019:03:37:59 +0000] "GET /images/drawings/fox_g9it7.jpg HTTP/1.1" 304 - "-" "Googlebot
66.249.69.31 - - [03/Mar/2019:03:37:59 +0000] "GET /images/drawings/fox_g9it7.jpg HTTP/1.1" 304 - "-" "Googlebot
122.114.127.156 - - [03/Mar/2019:03:41:54 +0000] "GET /user.php?act=login HTTP/1.1" 302 295 "554fcae493e564ee0dc
122.114.127.156 - - [03/Mar/2019:03:41:54 +0000] "GET /user.php?act=login HTTP/1.1" 302 295 "554fcae493e564ee0dc
122.114.127.156 - - [03/Mar/2019:03:41:56 +0000] "GET / HTTP/1.1" 200 57 "554fcae493e564ee0dc75bdf2ebf94caads|a:
122.114.127.156 - - [03/Mar/2019:03:41:56 +0000] "GET / HTTP/1.1" 200 57 "554fcae493e564ee0dc75bdf2ebf94caads|a:
122.114.127.156 - - [03/Mar/2019:03:41:57 +0000] "GET /user.php?act=login HTTP/1.1" 302 295 "45ea207d7a2b68c4958
122.114.127.156 - - [03/Mar/2019:03:41:57 +0000] "GET /user.php?act=login HTTP/1.1" 302 295 "45ea207d7a2b68c4958
122.114.127.156 - - [03/Mar/2019:03:41:57 +0000] "GET / HTTP/1.1" 200 57 "45ea207d7a2b68c49582d2d22adf953aads|a:
122.114.127.156 - - [03/Mar/2019:03:41:57 +0000] "GET / HTTP/1.1" 200 57 "45ea207d7a2b68c49582d2d22adf953aads|a:
59.56.241.218 - - [03/Mar/2019:03:42:35 +0000] "GET / HTTP/1.1" 200 5087 "-" "Mozilla/5.0 (Windows NT 6.1) Apple
59.56.241.218 - - [03/Mar/2019:03:42:35 +0000] "GET / HTTP/1.1" 200 5087 "-" "Mozilla/5.0 (Windows NT 6.1) Apple
52.53.201.78 - - [03/Mar/2019:03:43:05 +0000] "GET / HTTP/1.1" 200 6049 "-" "Mozilla/5.0 (Macintosh; Intel Mac O
52.53.201.78 - - [03/Mar/2019:03:43:05 +0000] "GET / HTTP/1.1" 200 6049 "-" "Mozilla/5.0 (Macintosh; Intel Mac O
34.235.233.220 - - [03/Mar/2019:03:50:49 +0000] "GET / HTTP/1.1" 200 13894 "-" "Mozilla/5.0 (Windows NT 10.0; Wi
34.235.233.220 - - [03/Mar/2019:03:50:49 +0000] "GET / HTTP/1.1" 200 13894 "-" "Mozilla/5.0 (Windows NT 10.0; Wi
3.89.137.200 - - [03/Mar/2019:03:52:25 +0000] "GET /california/homeschool_resources.html HTTP/1.1" 200 16335 "-"
3.89.137.200 - - [03/Mar/2019:03:52:25 +0000] "GET /california/homeschool_resources.html HTTP/1.1" 200 16335 "-"
148.251.195.14 - - [03/Mar/2019:03:55:23 +0000] "GET /robots.txt HTTP/1.1" 302 292 "-" "Mozilla/5.0 (compatible;
148.251.195.14 - - [03/Mar/2019:03:55:23 +0000] "GET /robots.txt HTTP/1.1" 302 292 "-" "Mozilla/5.0 (compatible;
125.160.139.255 - - [03/Mar/2019:03:55:23 +0000] "GET / HTTP/1.1" 200 184 "-" "Mozilla/5.0 (Windows NT 6.1; WOW6
125.160.139.255 - - [03/Mar/2019:03:55:23 +0000] "GET / HTTP/1.1" 200 184 "-" "Mozilla/5.0 (Windows NT 6.1; WOW6
148.251.195.14 - - [03/Mar/2019:03:55:25 +0000] "GET / HTTP/1.1" 200 57 "-" "Mozilla/5.0 (compatible; MJ12bot/v1
148.251.195.14 - - [03/Mar/2019:03:55:25 +0000] "GET / HTTP/1.1" 200 57 "-" "Mozilla/5.0 (compatible; MJ12bot/v1
148.251.195.14 - - [03/Mar/2019:03:55:26 +0000] "GET /robots.txt HTTP/1.1" 302 296 "-" "Mozilla/5.0 (compatible;
The following questions are "as recorded in this log."
1) Save the data set as a file
vi lab01.log
i ### insert mode
paste the data
press [ESC]
:wq ### write and quit the file
2) How many diffent visitors were recorded to the web server?
3) How many times did each visitor IP Address access websites on this server?
4) How many times was the request valid (200-series reponse)
5) How many times was the request redirected (300-series response)
6) How many times did robots scrape webpages on the server?
7) How many times did hackers try to login to websites on the server?
8) How would you sort just the visitor IP and their browser?
Thank you for dropping by!
Please join me:
/ paulwpoteete
/ paulwpoteete
/ paulwpoteete
/ paulwpoteete
https://plus.google.com/+PaulWPoteete
/ paulwpoteete
#linux, #unix, #tutorial, #administration, #engineering, #beginner, #expert, #advanced, #intermediate, #training
En esta página del sitio puede ver el video en línea Tutorial: How to Perform Data Analysis on Linux and UNIX de Duración hora minuto segunda en buena calidad , que subió el usuario The Expat Professor 07 marzo 2019, comparta el enlace con amigos y conocidos, en youtube este video ya ha sido visto 1,846 veces y le gustó like a los espectadores. Disfruta viendo!
