Migrate data from one collection to other

At the moment there is no command in MongoDB that would do this.

db.<collection_name>.find().forEach(function(d){ db.getSiblingDB('<new_database>')['<collection_name>'].insert(d); });

Quick and easy way to create a collection on mongodb

 

mongosh mongodb://admin:pass@localhost:27017

use sandbox

db.createCollection("synthetic");

db.synthetic.insertOne({ message: "Synthetic database initialized", timestamp: new Date() })

use admin

db.createUser({ user: "sandbox", pwd: "PassSand", roles: [ { role: "readWrite", db: "sandbox" } ] });

Enable SNMP on Ubuntu Server

On the agent and manager server:

apt update

apt install snmp snmp-mibs-downloader

On the agent server:

vim /etc/snmp/snmp.conf  

/etc/snmp/snmp.conf

# As the snmp packages come without MIB files due to license reasons, loading # of MIBs is disabled by default. If you added the MIBs you can reenable # loading them by commenting out the following line.  

#mibs :

vim /etc/snmp/snmpd.conf

Comment the following lines: 

 #  Listen for connections from the local system only
#agentAddress  udp:127.0.0.1:161
#  Listen for connections on all interfaces (both IPv4 *and* IPv6)
agentAddress udp:161,udp6:[::1]:161

vim /etc/snmp/snmpd.conf

Add these lines: 

createUser bootstrap MD5 temp_password DES

rwuser bootstrap priv
rwuser demo priv 

systemctl restart snmpd

On the Manager server:

Verify with snmpget 

snmpget -u bootstrap -l authPriv -a MD5 -x DES -A temp_password -X temp_password agent_server_ip_address 1.3.6.1.2.1.1.1.0 

Output
SNMPv2-MIB::sysDescr.0 = STRING: Linux agent 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64 

Create a user based on bootstrap 

snmpusm -u bootstrap -l authPriv -a MD5 -x DES -A temp_password -X temp_password agent_server_ip_address create demo bootstrap 

 Change the password

snmpusm -u demo -l authPriv -a MD5 -x DES -A temp_password -X temp_password agent_server_ip_address passwd temp_password new_password 

On the Agent server:

Remove the bootstrap

vim /etc/snmp/snmpd.conf 

Comment the following lines:

...
#createUser bootstrap MD5 temp_password DES
#rwuser bootstrap priv
...

On the Manager server:

snmpusm -u bootstrap -l authPriv -a MD5 -x DES -A temp_password -X temp_password 100.90.8.48 delete bootstrap
User successfully deleted.

 
On the Agent server: 

systemctl restart snmpd
 

How to Run Linux Commands in Background

 Using Ampersand
[command] &
 

Check the background processes' status in the current shell session with the jobs command jobs

Using System Redirects [command] output.log>&1 &
ping phoenixnap.com >/dev/null 2>&1 &

Using nohup
nohup myscript.sh >myscript.log 2>&1 </dev/null &

Bash script do a cleanup on exit

Excellent way to do a cleanup on exit.

#!/bin/bash

set -e # Exit on error

# Functions for cleanup on exit

cleanup() {

echo "Removing what is done..."

}

trap cleanup EXIT

Install grafana

# sudo apt-get install -y apt-transport-https software-properties-common wget
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
wget is already the newest version (1.21.4-1ubuntu4.1).
wget set to manually installed.
The following NEW packages will be installed:
  apt-transport-https
The following packages will be upgraded:
  python3-software-properties software-properties-common
2 upgraded, 1 newly installed, 0 to remove and 54 not upgraded.
Need to get 48.1 kB of archives.
After this operation, 35.8 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu noble/universe amd64 apt-transport-https all 2.7.14build2 [3,974 B]
Get:2 http://us.archive.ubuntu.com/ubuntu noble-updates/main amd64 software-properties-common all 0.99.49.1 [14.4 kB]
Get:3 http://us.archive.ubuntu.com/ubuntu noble-updates/main amd64 python3-software-properties all 0.99.49.1 [29.7 kB]
Fetched 48.1 kB in 0s (180 kB/s)                    
Selecting previously unselected package apt-transport-https.
(Reading database ... 83695 files and directories currently installed.)
Preparing to unpack .../apt-transport-https_2.7.14build2_all.deb ...
Unpacking apt-transport-https (2.7.14build2) ...
Preparing to unpack .../software-properties-common_0.99.49.1_all.deb ...
Unpacking software-properties-common (0.99.49.1) over (0.99.48) ...
Preparing to unpack .../python3-software-properties_0.99.49.1_all.deb ...
Unpacking python3-software-properties (0.99.49.1) over (0.99.48) ...
Setting up apt-transport-https (2.7.14build2) ...
Setting up python3-software-properties (0.99.49.1) ...
Setting up software-properties-common (0.99.49.1) ...
Processing triggers for man-db (2.12.0-4build2) ...
Processing triggers for dbus (1.14.10-4ubuntu4.1) ...
Scanning processes...                                                                                                                                                                    
Scanning linux images...                                                                                                                                                                 

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host

 

# sudo mkdir -p /etc/apt/keyrings/
wget -q -O - https://apt.grafana.com/gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null
root@pb-grafana-d001:~# echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main
root@pb-grafana-d001:~# echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com beta main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com beta main
root@pb-grafana-d001:~# # Updates the list of available packages
sudo apt-get update
Hit:1 http://us.archive.ubuntu.com/ubuntu noble InRelease
Hit:2 http://us.archive.ubuntu.com/ubuntu noble-updates InRelease                                                                                         
Get:3 https://apt.grafana.com stable InRelease [7,660 B]                                                                                                  
Hit:4 http://us.archive.ubuntu.com/ubuntu noble-backports InRelease                                   
Hit:5 http://security.ubuntu.com/ubuntu noble-security InRelease                          
Get:6 https://pkgs.tailscale.com/stable/ubuntu noble InRelease      
Get:7 https://apt.grafana.com beta InRelease [5,975 B]
Get:8 https://apt.grafana.com stable/main amd64 Packages [326 kB]
Get:9 https://apt.grafana.com beta/main amd64 Packages [1,616 B]
Fetched 348 kB in 1s (485 kB/s)
Reading package lists... Done
root@pb-grafana-d001:~# # Installs the latest OSS release:
sudo apt-get install grafana
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  musl
The following NEW packages will be installed:
  grafana musl
0 upgraded, 2 newly installed, 0 to remove and 54 not upgraded.
Need to get 127 MB of archives.
After this operation, 471 MB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://us.archive.ubuntu.com/ubuntu noble/universe amd64 musl amd64 1.2.4-2 [416 kB]
Get:2 https://apt.grafana.com stable/main amd64 grafana amd64 11.4.0 [127 MB]
Fetched 127 MB in 3s (45.8 MB/s)  
Selecting previously unselected package musl:amd64.
(Reading database ... 83699 files and directories currently installed.)
Preparing to unpack .../musl_1.2.4-2_amd64.deb ...
Unpacking musl:amd64 (1.2.4-2) ...
Selecting previously unselected package grafana.
Preparing to unpack .../grafana_11.4.0_amd64.deb ...
Unpacking grafana (11.4.0) ...
Setting up musl:amd64 (1.2.4-2) ...
Setting up grafana (11.4.0) ...
info: Selecting UID from range 100 to 999 ...

info: Adding system user `grafana' (UID 110) ...
info: Adding new user `grafana' (UID 110) with group `grafana' ...
info: Not creating home directory `/usr/share/grafana'.
### NOT starting on installation, please execute the following statements to configure grafana to start automatically using systemd
 sudo /bin/systemctl daemon-reload
 sudo /bin/systemctl enable grafana-server
### You can start grafana-server by executing
 sudo /bin/systemctl start grafana-server
Processing triggers for man-db (2.12.0-4build2) ...
Scanning processes...                                                                                                                                                                    
Scanning linux images...                                                                                                                                                                 

Running kernel seems to be up-to-date.

No services need to be restarted.

No containers need to be restarted.

No user sessions are running outdated binaries.

No VM guests are running outdated hypervisor (qemu) binaries on this host.

 

 

# sudo systemctl daemon-reload
sudo systemctl start grafana-server

 

# sudo systemctl status grafana-server
● grafana-server.service - Grafana instance
     Loaded: loaded (/usr/lib/systemd/system/grafana-server.service; disabled; preset: enabled)
     Active: active (running) since Mon 2024-12-09 21:12:22 UTC; 6s ago
       Docs: http://docs.grafana.org
   Main PID: 2728 (grafana)
      Tasks: 18 (limit: 962)
     Memory: 122.7M (peak: 143.1M swap: 1.7M swap peak: 1.7M)
        CPU: 1.368s
     CGroup: /system.slice/grafana-server.service
             └─2728 /usr/share/grafana/bin/grafana server --config=/etc/grafana/grafana.ini --pidfile=/run/grafana/grafana-server.pid --packaging=deb cfg:default.paths.logs=/var/log/gr>

Dec 09 21:12:28 pb-grafana-d001 grafana[2728]: logger=migrator t=2024-12-09T21:12:28.43369355Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v>
Dec 09 21:12:28 pb-grafana-d001 grafana[2728]: logger=migrator t=2024-12-09T21:12:28.434410666Z level=info msg="Migration successfully executed" id="create index UQE_dashboard_public_c>
Dec 09 21:12:28 pb-grafana-d001 grafana[2728]: logger=migrator t=2024-12-09T21:12:28.444885735Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id>
Dec 09 21:12:28 pb-grafana-d001 grafana[2728]: logger=migrator t=2024-12-09T21:12:28.445705164Z level=info msg="Migration successfully executed" id="create index IDX_dashboard_public_c>
Dec 09 21:12:28 pb-grafana-d001 grafana[2728]: logger=migrator t=2024-12-09T21:12:28.456330875Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2"
Dec 09 21:12:28 pb-grafana-d001 grafana[2728]: logger=migrator t=2024-12-09T21:12:28.457375016Z level=info msg="Migration successfully executed" id="drop index UQE_dashboard_public_con>
Dec 09 21:12:28 pb-grafana-d001 grafana[2728]: logger=migrator t=2024-12-09T21:12:28.469944357Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_d>
Dec 09 21:12:28 pb-grafana-d001 grafana[2728]: logger=migrator t=2024-12-09T21:12:28.470633922Z level=info msg="Migration successfully executed" id="drop index IDX_dashboard_public_con>
Dec 09 21:12:28 pb-grafana-d001 grafana[2728]: logger=migrator t=2024-12-09T21:12:28.489819777Z level=info msg="Executing migration" id="Drop public config table"
Dec 09 21:12:28 pb-grafana-d001 grafana[2728]: logger=migrator t=2024-12-09T21:12:28.490410095Z level=info msg="Migration successfully executed" id="Drop public config table" duration=>

 

# sudo systemctl enable grafana-server.service
Synchronizing state of grafana-server.service with SysV service script with /usr/lib/systemd/systemd-sysv-install.
Executing: /usr/lib/systemd/systemd-sysv-install enable grafana-server
Created symlink /etc/systemd/system/multi-user.target.wants/grafana-server.service → /usr/lib/systemd/system/grafana-server.service.

Install prometheus

Download prometheus

# wget https://github.com/prometheus/prometheus/releases/download/v3.0.1/prometheus-3.0.1.linux-amd64.tar.gz
--2024-12-09 22:12:17--  https://github.com/prometheus/prometheus/releases/download/v3.0.1/prometheus-3.0.1.linux-amd64.tar.gz
Resolving github.com (github.com)... 140.82.113.4
Connecting to github.com (github.com)|140.82.113.4|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/6838921/d29591b9-3d89-4c1b-85d3-0c4c38069329?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20241209%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241209T221217Z&X-Amz-Expires=300&X-Amz-Signature=6baf78ee0e923cc1f36128a8b4a85d8f03617a0804b49256d062adfbb0cd05fd&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dprometheus-3.0.1.linux-amd64.tar.gz&response-content-type=application%2Foctet-stream [following]
--2024-12-09 22:12:17--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/6838921/d29591b9-3d89-4c1b-85d3-0c4c38069329?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=releaseassetproduction%2F20241209%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20241209T221217Z&X-Amz-Expires=300&X-Amz-Signature=6baf78ee0e923cc1f36128a8b4a85d8f03617a0804b49256d062adfbb0cd05fd&X-Amz-SignedHeaders=host&response-content-disposition=attachment%3B%20filename%3Dprometheus-3.0.1.linux-amd64.tar.gz&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.111.133, 185.199.110.133, 185.199.109.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 112995464 (108M) [application/octet-stream]
Saving to: ‘prometheus-3.0.1.linux-amd64.tar.gz’

prometheus-3.0.1.linux-amd64.tar.gz            100%[=================================================================================================>] 107.76M  63.1MB/s    in 1.7s    

2024-12-09 22:12:19 (63.1 MB/s) - ‘prometheus-3.0.1.linux-amd64.tar.gz’ saved [112995464/112995464]

 

tar over the package

# tar xvfz prometheus-*.tar.gz
prometheus-3.0.1.linux-amd64/
prometheus-3.0.1.linux-amd64/promtool
prometheus-3.0.1.linux-amd64/LICENSE
prometheus-3.0.1.linux-amd64/prometheus
prometheus-3.0.1.linux-amd64/prometheus.yml
prometheus-3.0.1.linux-amd64/NOTICE

Move to the directory and start prometheus

# cd prometheus-3.0.1.linux-amd64/

# ./prometheus --config.file=./prometheus.yml &

Confirm that Prometheus is running by navigating to http://localhost:9090

Create a database with MongoDB

Connect to mongodb

root@mongo:/home/mongo# mongosh "mongodb://admin:adminpassword@localhost:27017"

Create the database, just command use

test> use analytics
switched to db analytics

 

Create the collection
analytics> db.createCollection("traffic"); 

 

Insert one record on the collection

analytics> db.traffic.insertOne({ message: "Analytics database initialized", timestamp: new Date() })

Use admin database

analytics> use admin
switched to db admin

 

Create a user on the database admin
admin> db.createUser({ user: "userName", pwd: "passwdthatyouwant", roles: [{ role: "readWrite", db: "DBthatyouwant" }] });
{ ok: 1 }

Azure VM Standard BS1 mods, swap and lvm

 The azure Azure VM Standard BS1 has 65GB of hard disk for the operative system. But there is an issue with redhat running on 1GB, it crashes, and the other one is that lvm doesn't leverage the space.

The solution create a swap and extend the lvm!

 

Creation of swap

fallocate -l 3G /swap-file
chmod 600 /swap-file
mkswap /swap-file
swapon /swap-file
swapon --show
cp /etc/fstab /etc/fstab.bak
echo '/swap-file none swap sw 0 0' | sudo tee -a /etc/fstab

 

Check the lsblk and extend the partition

lsblk

lvextend --size +10GB --resizefs /dev/rootvg/rootlv

df -h

Rsyslog with high resolution timestamp

 

By default, rsyslog uses traditional timestamp, which in date command's format would be:

%b %d %H:%M:%S

This is enabled by the following line in /etc/rsyslog.conf:

$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat 

To enable high precision timestamping, comment out the line:

# $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat 

which will make rsyslog timestamping in the RFC 3339 format.

rsyslog

 global(
  workDirectory="/var/spool/rsyslog"
)

# Load the imfile module to read logs from a file
module(load="imtcp" MaxSessions="500")
input(type="imtcp" port="514")

template(name="probe-request" type="list" option.jsonf="on") {
         property(outname="@timestamp" name="timereported" dateFormat="rfc3339" format="jsonf")
         property(outname="host" name="hostname" format="jsonf")
         property(outname="sourceaddress" name="msg" regex.type="ERE" regex.submatch="1" regex.expression="SA:([^[:space:]]*)" regex.nomatchmode="BLANK" format="jsonf")
         property(outname="signal" name="msg" field.number="10" field.delimiter="32" format="jsonf" onEmpty="null")
         property(outname="wifi" name="msg" regex.type="ERE" regex.submatch="1" regex.expression="Probe Request \\((.*?)\\)" regex.nomatchmode="BLANK" format="jsonf")
}

template(name="roamed" type="list" option.jsonf="on") {
         property(outname="@timestamp" name="timereported" dateFormat="rfc3339" format="jsonf")
         property(outname="host" name="hostname" format="jsonf")
         property(outname="sourceaddress" name="msg" field.number="18" field.delimiter="32" format="jsonf" onEmpty="null")
         property(outname="signal" name="msg" field.number="10" field.delimiter="32" format="jsonf" onEmpty="null")
         property(outname="wifi" name="msg" field.number="21" field.delimiter="32" format="jsonf")
}


# Send logs with the specified tag to the console
if $msg contains 'Probe Request (' then {
        action(type="omfile" file="/var/log/syslogs/unifi-probe.log" template="probe-request")
}

if $msg contains 'roamed' then {
        action(type="omfile" file="/var/log/syslogs/unifi-roamed.log" template="roamed")
}

Use pip in a virtual environment

 

If you haven't yet learned a tool to set up a virtual environment, I highly recommend it. All Python programmers should learn one. I recommend venv or virtualenv to beginners. To install venv, run:

sudo apt install python3-venv

Then create a virtual environment in your project directory like this:

 python3 -m venv .venv

Now activate your virtual environment by running:

 source .venv/bin/activate

This modifies your PATH environment variable to include .venv/bin/. Now you can install PyPI packages using pip into your virtual envirnoment (in this case .venv/), like this:

pip install requests

If you don't want to activate and deactivate virtual environments, you can run pip and python directly from the virtual environment, like this:

$ .venv/bin/pip install requests
$ .venv/bin/python
>>> import requests
>>> 

 https://stackoverflow.com/questions/75608323/how-do-i-solve-error-externally-managed-environment-every-time-i-use-pip-3

Razr Kiyo on Ubuntu

 Woow I am very happy with this camera. 

 

 

It worked as soon I connected. The only issue was the zoom. So, I should run a couple of commands, you can crop the video with these:

sudo apt-get install v4l-utils

 

List the devices:

alejandro@minipc:~$ v4l2-ctl --list-devices
Razer Kiyo (usb-0000:00:14.0-3):
    /dev/video0
    /dev/video1
    /dev/media0
 

See for the options of the device video0

alejandro@minipc:~$ v4l2-ctl -d /dev/video0 --list-ctrls

User Controls

                     brightness 0x00980900 (int)    : min=0 max=255 step=1 default=128 value=128
                       contrast 0x00980901 (int)    : min=0 max=255 step=1 default=128 value=128
                     saturation 0x00980902 (int)    : min=0 max=255 step=1 default=128 value=128
        white_balance_automatic 0x0098090c (bool)   : default=1 value=1
                           gain 0x00980913 (int)    : min=0 max=255 step=1 default=0 value=0
           power_line_frequency 0x00980918 (menu)   : min=0 max=2 default=2 value=1 (50 Hz)
      white_balance_temperature 0x0098091a (int)    : min=2000 max=7500 step=10 default=4000 value=5630 flags=inactive
                      sharpness 0x0098091b (int)    : min=0 max=255 step=1 default=128 value=128
         backlight_compensation 0x0098091c (int)    : min=0 max=1 step=1 default=0 value=0

Camera Controls

                  auto_exposure 0x009a0901 (menu)   : min=0 max=3 default=3 value=3 (Aperture Priority Mode)
         exposure_time_absolute 0x009a0902 (int)    : min=3 max=2047 step=1 default=127 value=156 flags=inactive
     exposure_dynamic_framerate 0x009a0903 (bool)   : default=0 value=1
                   pan_absolute 0x009a0908 (int)    : min=-36000 max=36000 step=3600 default=0 value=0
                  tilt_absolute 0x009a0909 (int)    : min=-36000 max=36000 step=3600 default=0 value=0
                 focus_absolute 0x009a090a (int)    : min=0 max=255 step=1 default=0 value=83
     focus_automatic_continuous 0x009a090c (bool)   : default=1 value=0
                  zoom_absolute 0x009a090d (int)    : min=100 max=140 step=10 default=100 value=130
 

 The zoom_absolute value is 130, with 100 will be the minimum!

alejandro@minipc:~$ v4l2-ctl -d /dev/video0 -c zoom_absolute=100 

Now much better!