Skip to content

Container management

Jump to bottom
gabx edited this page Aug 28, 2020 · 33 revisions
# Container management

NOTES: since Fedora 22, yum is depreciated. Use dnf to manage packages.

Fedora search packages Fedora rpm packages

1. Systemd

All basic commands, like start, enable and watch a service can be found on this digital ocean tutorial.

1.1. Drop-in snippets

To create drop-in snippets for the unit file /usr/lib/systemd/system/unit, create the directory /etc/systemd/system/unit.d/ and place .conf files there to override or add new options. systemd will parse these .conf files and apply them on top of the original unit.

The easiest way to do this is to run:

# systemctl edit MyUnit

This opens the file /etc/systemd/system/unit.d/override.conf in your text editor (creating it if necessary) and automatically reloads the unit when you are done editing.

1.2. Environment variables

Systemd does not inherit from system environment variables. To list and set systemd specific variables:

$ systemctl show-environment
# systemctl set-environment MyVariable=MyValue
  • view the content of a unit file and all associated drop-in snippets.
$ systemctl cat MyUnit

To run a program or a command in a modified environment, use the $ env command.

$ env [OPTION]... [NAME=VALUE]... [COMMAND [ARG]...]

1.3. Useful systemd commands

  • list ports
# netstat -nltp
  • reload unit configuration
# systemctl reload MyUnit
  • make an unit impossible to start
# systemctl mask MyUnit
  • Create drop-in files
# systemctl edit MyUnit

This opens the file /etc/systemd/system/unit.d/override.conf in your text editor (creating it if necessary) and automatically reloads the unit when you are done editing.

  • Replace part of unit file

Create a drop-in file as explained above and write your changes: add, remove or replace. In case of replacement, the argument must be cleared before. Ex:

[Service]
ExecStart=
ExecStart=new command
  • Get current targets
$ systemctl list-units --type=target
  • Delete journal entries older than 2 weeks
# journalctl --vacuum-time=2weeks
  • show journal error, critical and priority messages
# journalctl -p err..alert
  • Show all messages since 20 minutes ago
# journalctl --since "20 min ago"

2. useful Linux commands

  • find a file :
# find myPath -iname 'myFile' -print
  • show directory structure and files
$ tree -a -L 'number' /path/
  • List running daemons
$ systemctl list-units --state=running | awk '{print $1}' | grep service > daemon.list
  • generate a MD5 based BSD password with algorithm 1:
$ openssl passwd -1
  • list socket and listening process
$ ss
  • get picture information
$ identify MyPic.jpg
MyPic.jpg JPEG 800x600 800x60B 112KB 0.000u 0:00.000

Identify is part of Image magic

  • list diff between two folders
$ diff -ENwbur dir1 dir2
  • remount directory as read/write. This can be useful for everything in the /proc/sys as it is mounted as RO.
# mount /proc/sys -o rw,remount
  • Files/dir modified in the last X days
find /dir -mtime -X -ls
  • get owner/permissions for a folder and its hierarchy
# namei -l /opt/mattermost/webapp/dist
f: /opt/mattermost/webapp/dist
dr-xr-xr-x root       root       /
drwxr-xr-x root       root       opt
drwxr-xr-x mattermost mattermost mattermost
drwxr-xr-x mattermost nginx      webapp
drwxr-xr-x mattermost nginx      dist

3. Network management

3.1. Basic linux commands

  • list ssh connected users:
# who -a
or
# netstat -tnpa | grep 'ESTABLISHED.*sshd'
  • see live traffic:
# tcpdump
  • list ports with connection:
# netstat -tnpa

See here for more commands to monitor the server traffic.

  • explore local network
# nmap 10.42.0.0/32 

Starting Nmap 7.60 ( https://nmap.org ) at 2018-04-08 15:15 UTC
Nmap scan report for 10.42.0.0
Host is up (0.000021s latency).
Not shown: 997 closed ports
PORT     STATE SERVICE
80/tcp   open  http
443/tcp  open  https
9099/tcp open  unknown

Nmap done: 1 IP address (1 host up) scanned in 1.69 seconds

3.2. Netstat

Netstat is a command line utility that can be used to list out all the network (socket) connections on a system. It lists out all the tcp, udp socket connections and the unix socket connections.

Apart from connected sockets it can also list listening sockets that are waiting for incoming connections. So by verifying an open port 80 you can confirm if a web server is running on the system or not. This makes it a very useful tool for network and system administrators.

See this article for a list of useful commands.

4. Cockpit

Cockpit is a new web based server manager to administer Linux server.

Point your browser to the following url: http://192.168.1.94:9090 to access the console locally.

4.1. configuration

  • Cockpit can be configured via /etc/cockpit/cockpit.conf. This file is not created at install.

Add user access only for cockpit-ws group.

Normally cockpit-ws uses cockpit-session and PAM to authenticate the user and start a user session.

  • The cockpit-ws program is the web service component. The service and its socket are started by systemd.To specify the TLS certificate the web service should use, simply drop a file with the extension .cert in the /etc/cockpit/ws-certs.d directory.

5. Package management

dnf is now the default package manager for fedora. Most commands are trivial.

TIP: when installing a package but not its dependencies, do the following:

  • download the rpm packages

  • run # rpm -ihv --force MyPackage.rpm --nodeps

  • install a package

# dnf install MyPackages
  • upgrade all packages
# dnf upgrade
  • get info about a packages
$ dnf info MyPackage
  • find which package a file belongs to
$ dnf provides /path/to/file
$ rpm -qf /path/to/file --queryformat '%{NAME}\n'
  • find which package a command belongs to
$ rpm -qf $(which foo)
  • find what a package provides
$ rpm -qp --provides file.rpm
  • find what a package requires
rpm -qp --requires file.rpm
  • list all new configuration files from upgrades
# rpmconf -a
  • List installed packages with no version number, then install from the list:
# dnf repoquery --qf "%{name}" --userinstalled > userinstalled.txt
# dnf install $(cat userinstalled.txt)

NOTE: when a configuration file is upgraded, new one is installed as myconf.rpmnew. Best practice:

  • save myconf
  • vimdiff the old and new config files and save to myconf
  • test
  • if everything is OK, remove both myconf.rpmnew,myconf.bck

5.1. Upgrade version

# dnf system-upgrade download --refresh --releasever=XX
# dnf system-upgrade reboot

If some of your packages have unsatisfied dependencies, the upgrade will refuse to continue until you run it again with an extra --allowerasing option

5.2. Build .rpm packages

If no Fedora packages are found, have a look here and see if there is a .rpm package.

Follow these steps.

A makerpm user and its home directory and bash shell have been created, and this user has set up his Fedora certificates and such when running $ fedora-packager-setup.

1- log as makerpm user. It will give you bash shell and a clean environment.

2- cd to rpmbuild

3- mkdir a new directory with your package name and cd to it. run $ rpmdev-setuptree

4- write the .spec file, place it in SPEC folder, run $ spectool -g foo.spec to download all needed sources and move them into SOURCESfolder

5- run $ fedpkg --dist fXX local

For intermediate builds:

$ fedpkg --dist fXX prep

$ fedpkg --dist fXX compile

$ fedpkg --dist fXX install

$ fedpkg --dist fXX lint

TIP:

  • download the source package, run
$ spectool -g foo.spec
  • write changelog with vim
$ vim MySpecfile.spec

then enter \c in vim when in the changelog section.

  • extract .rpm file
$ rpm2cpio MyApp.rpm | cpio -idmv
  • test a macro (example _sysuserdir)
$ rpm -E %_sysusersdir
/usr/lib/sysusers.d

5.2.1. Testings

All the below tests shall be successful and end with a zero status

  • test the %prep section
$ fedpkg --dist f24 prep
  • Locally test run of rpmbuild producing binary RPMs.
$ fedpkg local
  • generate a mock config
$ fedpkg mock-config

$ fedpkg --dist f24 lint
  • test macro definition
$ rpm -E %MyMacro

5.2.2. Create a local repo for custom rpm packages

To solve dependency issues, all custom rpm packages have to be listed in a repo. We will use here a local one sync with sqlite.

1- place all rpm in /storage/development/rpm

2- check everything is owned by root

3- run as root createrepo /storage/development/rpm. It will create and populate a subdirectory repodata.

4- run as root dnf config-manager add-repo file:///storage/development/rpm/. A repo file will be added in /etc/yum.repos.d

6- check your repo exists and can be read:

$ dnf repolist

7- resync the data base wnen installing package:

# dnf --refresh install MyPackage

NOTE : please visit Copr page. Copr is a Community projects build service that builds your open-source project and creates your own RPM repository.

6. SSH

6.1. First time login

  • be sure your are registered as user on the server
  • ask administrator to allow password auth
  • generate a SSH rsa key· if not already done
  • copy your key to the server:
ssh-copy-id [email protected]

You will be prompted to enter your user password. The ~/.ssh/authorized_keys file will then be created.Then,

$ chmod 600 ~/.ssh/authorized_keys
$ chmod 700 ~/.ssh

Connect to poppy:

$ ssh [email protected]

6.2. Port forwarding

Local forwarding allows you to tunnel a port available via the remote system through your ssh connection. The port appears as a local port on your system (thus “local forwarding.”).

Let’s say your web app is running on port 8000 on the remote.example.com box. To locally forward that system’s port 8000 to your system’s port 8000, use the -L option with ssh when you start your session:

ssh -L 8000:localhost:8000 remote.example.com

Once the ssh session is ready, keep it open, and you can type http://localhost:8000 in your browser to see your web app. The traffic between systems now travels securely over an ssh tunnel!

6.3. SCP

scp stands for secure cp (copy), which means you can copy files across ssh connection. You can use scp to copy files from or to a remote server

Examples:

  • Copy one single local file to a remote destination
$ scp /path/to/source-file user@host:/path/to/destination-folder/
  • Copy all files in a folder to a remote server
$ scp /path/to/folder/* user@server:/path/to/folder/

7. User management

  • list users
$ cut -d: -f1 /etc/passwd

7.1. add regular user and password

# useradd -G wheel -s /usr/bin/zsh UserName
# passwd UserName
Changing password for user UserName
New password:
Retype new password:

7.2. add an user to a group

# gpasswd --add username group

7.3. add a system user

This command will create a system user with no home directory, no group with same name as the user and its numeric identifier chosen in the SYS_UID_MIN-SYS_UID_MAX range.

# useradd -r -N -G MyGroup -s /sbin/nologin SystemUserName

NOTES:

  • When the login shell is intended to be non-functional, for example when the user account is created for a specific service, /usr/bin/nologin may be specified in place of a regular shell to politely refuse a login (see nologin(8)).
  • when shell option is added, please verify it is list in /etc/shells
  • Members of group wheel can run the sudo command and gain privileges.

7.4. Strong password policy

We are using PAM (Pluggable Authentication Modules)

7.4.1. Prevent reusing old passwords

# vim /etc/pam.d/system-auth 
password   sufficient   pam_unix.so sha512 shadow nullok try_first_pass use_authtok remember=5

Adding remember=5 prevent the user to reuse five most recently used passwords

7.4.2. Set Minimum Password Length

# vim /etc/pam.d/system-auth 
password   requisite   pam_cracklib.so retry=3 difok=3 minlen=8

Add the above line to set a minimum length of 8 characters

7.4.3. Set Password Expiration Period

# vim /etc/login.defs 
PASS_MAX_DAYS   90
PASS_MIN_DAYS   0
PASS_MIN_LEN    8
PASS_WARN_AGE   7

7.4.4. Administrative access

When a user executes the su command, they are prompted for the root password and, after authentication, are given a root shell prompt.

Because this program is so powerful, we limit who has access to the command to the special administrative group called wheel. For this, we edit the /etc/sudoers and /etc/pam.d/su by uncommenting this line #auth required pam_wheel.so use_uid.

7.5. Policy rules

/etc/polkit-1/rules.d/50-default.rules

7.6. list connected users

# who -a

8. Cron jobs

Cronie is now depreciated in the benefit of systemd timer

# systemctl disable crond
Removed symlink /etc/systemd/system/multi-user.target.wants/crond.service.

9. Network

9.1. NTPD

The Network Time protocol daemon, or ntpd is the most common method to synchronize the software clock of a GNU/Linux system with internet time servers. Its installation and configuration is rather trivial.

Don't forget to open and monitor the udp prt 123.

9.2. Network status

$ ip a
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether fa:16:3e:fc:27:73 brd ff:ff:ff:ff:ff:ff
    inet 10.52.16.184/24 brd 10.52.16.255 scope global dynamic eth0
......

Local IP: 10.52.16.184/24

list of services running on your system on tcp and udp ports:

# netstat -lnptu

9.3. Monitor network performances

Fedora includes the Net-SNMP software suite, which includes a flexible and extensible Simple Network Management Protocol (SNMP) agent

10. Logs

10.1. Log path

For practical reasons, logs have been grouped as much as possible in the /storage/log directory.

10.2. Journalctl

Systemd provides a centralized management solution for logging all kernel and userland processes. The system that collects and manages these logs is known as the journal.

10.3. useful commands

  • delete old archives and keep only 5 last days
# journalctl --vacuum-time=5d
  • view from last entries
$ journalctl -r
  • view a specific unit
$ journalctl -u MyUnit
  • view boot kernel messages
$ journalctl -b

11. Ressource management

See how much RAM is available

$ free -m

$ iostat
Linux 4.0.1-1-apparmor (poppy) 	05/10/2015 	_x86_64_	(8 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           1.10    0.72    0.44    0.03    0.00   97.70

Device:            tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
sdc               0.98        10.02        15.28    1706795    2602976
sdb               3.60        17.63        67.03    3004719   11422208
sda               0.11         0.58         1.75      98808     298164
sdd               0.29         0.02        14.62       3643    2491468
dm-0              0.11         0.35         0.11      59753      18112
dm-1              0.01         0.19         1.64      32885     280044
dm-2              0.00         0.02         0.00       3613          4
dm-3              0.00         0.00         0.00        561          4
dm-4              0.00         0.00         0.00        456          0
dm-5              0.00         0.00         0.00        456          0
dm-6              0.00         0.00         0.00        456          0
bcache0           0.31         0.01        18.05       1320    3076480

11.1. Shared memory

Shared memory allows processes to access common structures and data by placing them in shared memory segments. It is the fastest form of inter-process communication available since no kernel involvement occurs when data is passed between the processes. In fact, data does not need to be copied between the processes.

Some parameters need to be configured.

A helper script /etc/scripts/shmsetup will define values for the system when run.

In the case of a Linux container, it will inherit from host values. So these are defined in our host machine.

11.1.1. SHMMAX Parameter

11.1.2. SHMMNI Parameter

11.1.3. SHMALL Parameter

12. Ressources

Clone this wiki locally