Category: Linux

Etcher, a popular open-source USB image writer tool for Windows, macOS and Linux, has just issued a new stable release.

Version 1.0 arrives almost one year to the day since we first introduced you to the easy-to-use image writer tool on this site. The stable release sees the app pick up a boat load of improvements that, its developers say, help make it “a much more stable and reliable tool”.

A Recap of Etcher Features

The company steering development of the app say over the course of the various beta releases Etcher was used to write over one million images to SD cards & USB drives.

Built using the Electron format, Etcher is a true cross-platform app that can write .iso, .img and .zip files to USB drives and SD cards.

The main interface is dead simple to use: you select an image, select a drive (the built in drive picker is designed to avoid you making mistakes and overwriting a hard drive, etc) and hit Flash. Validated burning double-checks images after writing so that you’re left faffing about trying to boot from a dud drive.

No sign of some previously planned features, like support for creating multi-boot USB sticks, or enabling persistent storage on Ubuntu images.

• Support for creating multi-boot USBs
• Support for persistent storage on Ubuntu images
• Registered Etcher as handler for *.img and *.iso files

It’s not just the GUI client that’s gotten an update though. Etcher 1.0 also sees the first experimental release of the Etcher CLI.

The Etcher CLI lets you to write images and validate flashes from the command line. As it doesn’t rely on the Electron framework, it’s a smaller download and install size. Its developers also tout the ability for users to write custom scripts using the CLI to “perform tasks such as multi-writes.”

How to install Etcher on Ubuntu

Etcher 1.0 is available to download for Windows, macOS and Linux from the Etcher.io website or from its Github page:

Etcher on Github

Linux builds are provided in the AppImage package format. App Images are self-contained runtimes that do not require manual installation or root (but does require you to give it the necessary permissions to run as a programme). They will run on pretty much every distro out there — just download, and double-click to run:

If you prefer to install your apps in a more traditional way, using apt, you can install Etcher on Ubuntu from the Etcher repository.

Now, getting this set up is a little bit more involved that with a regular PPA, but this methods the benefit of ensuring you get all future Etcher updates automatically through your update manager.

To add the Etcher repo open the Software & Updates app using the Unity Dash (or an alternative app launcher):

Select the ‘Other Software’ tab in Software & Updates [1]

Click ‘Add’ [2] and paste the following the entry field of the box that appears:

deb http://ift.tt/2qE7Sgv stable etcher

Click ‘Add Source’ [3] to confirm the change, then close Software & Updates. You’ll likely be prompted to update your software sources.

The next step is to add the repository key. This allows Ubuntu to verify that packages installed from the repository are made by who they say they are. You have to add this key to be able to install Etcher; Ubuntu will disable unsigned repos.

Open a new Terminal window, paste the following command, and then hit return/enter:

sudo apt-key adv --keyserver hkp://pgp.mit.edu:80 --recv-keys 379CE192D401AB61

Finally, update your packages list and install the app:

sudo apt update && sudo apt install etcher-electron

That’s it; launch Etcher from the Unity Dash (or an alternative app launcher) and follow the on-screen instructions.

H

ow do I run commands in parallel in a bash shell script running under Linux or Unix-like operating system? How can I run multiple programs in parallel from a bash script?

You have various options to run programs or commands in parallel:

=> Use GNU/parallel or xargs command.

=> Use wait built-in command with &.

=> Use xargs command.

Putting jobs in background

The syntax is:
command &
command arg1 arg2 &
custom_function &
OR
prog1 &
prog2 &
wait
prog3
In above code sample, prog1, and prog2 would be started in the background, and the shell would wait until those are completed before starting the next program named progr3.

Examples

In this following example run sleep command in the background:
$ sleep 60 &
$ sleep 90 &
$ sleep 120 &
To displays status of jobs in the current shell session run jobs command as follows:
$ jobs
Sample outputs:

[1]   Running                 sleep 60 &
[2]-  Running                 sleep 90 &
[3]+  Running                 sleep 120 &

Let us write a simple bash shell script:

#!/bin/bash
# Our custom function
cust_func(){
  echo "Do something $1 times..."
  sleep 1
}
# For loop 5 times
for i in {1..5}
do
	cust_func $i & # Put a function in the background
done
 
## Put all cust_func in the background and bash 
## would wait until those are completed 
## before displaying all done message
wait 
echo "All done"

Let us say you have a text file as follows:
$ cat list.txt
Sample outputs:

http://ift.tt/2piFZq2
http://ift.tt/2pKEy69
http://ift.tt/2pj1KWo
http://ift.tt/2pKCco9
http://ift.tt/2piQTMf
http://ift.tt/2pKvOgu
http://ift.tt/2piKKjd
http://ift.tt/2pKAL96
http://ift.tt/2piLzIT
http://ift.tt/2pKlQfe
http://ift.tt/2piXVAI
http://ift.tt/2pKvtKT

To download all files in parallel using wget:

#!/bin/bash
# Our custom function
cust_func(){
  wget -q "$1"
}
 
while IFS= read -r url
do
        cust_func "$url" &
done < list.txt
 
wait
echo "All files are downloaded."

GNU parallel examples

From the GNU project site:

GNU parallel is a shell tool for executing jobs in parallel using one or more computers. A job can be a single command or a small script that has to be run for each of the lines in the input. The typical input is a list of files, a list of hosts, a list of users, a list of URLs, or a list of tables.

The syntax is pretty simple:
parallel ::: prog1 prog2
For example, you can find all *.doc files and gzip (compress) it using the following syntax:
$ find . -type f -name '*.doc' | parallel gzip --best
$ find . -type f -name '*.doc.gz'
Our above wget example can be simplified using GNU parallel as follows:
$ cat list.txt | parallel -j 4 wget -q {}
OR
$ parallel -j 4 wget -q {} < list.txt

See also

G

oAccess is a real-time Apache/Nginx/Lighttpd web log analyzer and interactive viewer that runs in a terminal and provides fast and valuable HTTP statistics for system administrators that require a visual report on the fly. How do I install GoAccess on Ubuntu Linux server? How can I instal and use GoAccess on Linux or Unix-like system?

GoAccess is a free and open source real-time web log analyzer and interactive viewer that runs in a terminal in Linux/Unix/*BSD systems or through your browser. T

his tutorial shows how to install the GoAccess on Linux/Unix and how to use it on the *nix command line

.

Install GoAccess

You must install GoAccess as per your Linux or Unix distro.

Install GoAccess on Ubuntu Linux

Type the following apt-get command/apt command as follows:
$ sudo apt-get install goaccess
Sample outputs:

Install GoAccess on Debian Linux

Type the following apt-get command/apt command as follows:
$ sudo apt-get install goaccess

Install GoAccess on Debian Linux

Type the following apt-get command/apt command as follows:
$ sudo apt-get install goaccess

Install GoAccess on Alpine Linux

Type the following apk command as follows:
# apk add goaccess

Install GoAccess on CentOS Linux

First turn on the EPEL repo and type the following yum command as follows:
$ sudo yum install epel-release
$ sudo yum install goaccess

Install GoAccess on Fedora Linux

First turn on the EPEL repo and type the following dnf command as follows:
$ sudo dnf install epel-release
$ sudo dnf install goaccess

Install GoAccess on Arch Linux

Type the following pacman command as follows:
# pacman -S goaccess

Install GoAccess on FreeBSD UNIX

To install the port:
# cd /usr/ports/sysutils/goaccess/ && make install clean
OR To add the package:
# pkg install goaccess

Install GoAccess on macOS UNIX (Homebrew)

Type the following brew command:
$ brew install goaccess

Install GoAccess on OpenBSD UNIX

Type the following pkg_add command:
$ doas pkg_add goaccess

Installing GoAccess using source code method

Type the following command to download and compile GoAccess on Unix-like system:
$ cd /tmp
$ wget http://ift.tt/2oMTfXs
$ tar -zxvf goaccess-1.2.tar.gz
$ cd goaccess-1.2/
$ ./configure --enable-utf8 --enable-geoip=legacy && make
$ sudo make install

How do I use GoAccess?

The syntax is:
goaccess -f /path/to/nginx/access.log
goaccess -f /path/to/apache/access.log
goaccess -f /path/to/lighttpd/access.log
goaccess -f /path/to/lighttpd/access.log /path/to/lighttpd/access.log.1
zcat /path/to/nginx/access.log.*.gz | goaccess access.log -
goaccess [options] /path/to/lighttpd/access.log

Examples

Let us see some examples.

How can I see output on screen with a live report?

goaccess -f /var/log/nginx/access.log
You will be promoted to select “Log Format Configuration”:

Next you will see a report as follows:

Important shortcut keys

You can use the following keys:

• q – Quit the program.
• h or ? – See help.
• 0-9 and Shift + 0 – Set selected module to active.
• j – Scroll down within expanded module.
• k – Scroll up within expanded module.
• c – Set or change scheme color.
• ^f – Scroll forward one screen within active module.
• ^b – Scroll backward one screen within active module.
• TAB – Iterate modules (forward).
• SHIFT + TAB – Iterate modules (backward).
• s – Sort options for active module.
• /Search – across all modules (regex allowed).
• n – Find position of the next occurrence.
• g – Move to the first item or top of screen.
• G – move to the last item or bottom of screen.

How do I generate an HTML report?

The syntax is:
goaccess -f /var/log/nginx/access.log --log-format=COMBINED -o http://ift.tt/2oYHv05
Sample outputs:

Parsing... [669] [1112]

You can view report with your web-browser. Here is a sample report:
http://ift.tt/2oYOw13

How do I use goaccess over an ssh based session?

The syntax is:
$ ssh [email protected] 'cat /var/log/nginx/access.log'|goaccess --log-format=COMBINED -a -
$ ssh [email protected] 'cat /var/log/nginx/access.log'|goaccess --log-format=COMBINED -
$ ssh [email protected] 'cat /var/log/nginx/access.log'|goaccess --log-format=COMBINED -o http://ift.tt/2oYNodZ -

How do I view real time stats in browser?

Run it as follows:
$ goaccess -f access.log -o report.html --real-time-html --addr=192.168.1.254 --port=8022
See man page or project page for more info.