bash tip: collapse or parse a big text doc into individual sorted words from columns

Start with list.txt like:

server7858   server7858   server7858   server7858   server7861   server7860   server8310   server8310   server7863   server8311

server7859   server7859   server7859   server7859   server8781   server8676   server8677   server8677   server8679   server8782

Which has duplicates and long lines and crap. Run this:

rm list2.txt

rm list3.txt

for word in `cat list.txt`; do echo $word ; done |sort |uniq >> list2.txt

sed -ibak -e ‘s/ //g’ list2.txt

cat list2.txt |sort|uniq > list3.txt

vi list3.txt

ta da!

if you need word counts and such, pipe it through wc before running uniq

If you need to collapse multi line (multi-line multiline) data like this:

fldcvisla8524:

packages.MQSeriesServer.installdate: 1439579830

fldcvfsla13746:

packages.MQSeriesServer.installdate: 1486575523

Into:

Continue reading “bash tip: collapse or parse a big text doc into individual sorted words from columns”

Bash case switches

if [[ ${TAG_VALUE}XXX == "XXX" ]]; then

  echo "TAG_VALUE was blank, exiting!"

  exit

fi

case @option.Command@ in

  start)

    echo 'I am the first box'

    ;;

  stop)

    echo 'I am the second box'

    ;;

  restart)

    echo 'I am the third box'

    ;;

  status)

    echo 'I am the fourth box'

    ;;

  5)

    echo 'I am the fifth box'

    ;;

  *)

    echo 'I am another box higher than 5'

    ;;

esac

Run ssh commands remotely for one or many or a list of servers

single:

ssh -o “StrictHostKeyChecking no” -t ${server} ‘sudo sed -ibak -e ‘s#https://oldchef.server.com/#https://newchef.server.com/#g’ /etc/chef/client.rb’

many:

for server in server001 server002 server007; do echo ${server} & ssh -o “StrictHostKeyChecking no” -t ${server} ‘sudo sed -ibak -e ‘s#https://oldchef.server.com/#https://newchef.server.com/#g’ /etc/chef/client.rb’ ; done

list:

Continue reading “Run ssh commands remotely for one or many or a list of servers”

Comskip commercial skipping on plex

Plex doesn’t seem to install comskip on linux by default. Although it says it uses it, it’s not in the tarball so nothing actually happens. I guess it assumes you’ve figured it out on your own(?).

Install comskip:

Dependencies:

apt-get install -y autoconf libtool git build-essential libargtable2-dev libavformat-dev libsdl1.2-dev

sudo su –

git clone git://github.com/erikkaashoek/Comskip

cd Comskip

./autogen.sh

./configure

make

make install

Auto-skip *should* work now if you set it up in the DVR settings, existing recordings won’t have commercials skipped.

Manual commercial scans (i.e. go fix my existing recordings!):

Continue reading “Comskip commercial skipping on plex”

Making iTerm 2 highlight errors all the time

This helps when you have to chase down stupid stuff all day, every day:

Launch iTerm2

Go to iTerm2 – Preferences – Profiles – Advanced – Triggers – Edit

Click +

In the Regex, type:   (error|ERROR|Error)

Action: Highlight Text

Pick a color, Pick a background color that stand out for you

Check the Instant box

Close back out

I have:

(Fail|FAIL|fail)

(Differ|DIFFER|differ)

(Insufficient|INSUFFICIENT|insufficient)

(Access|ACCESS|access)

(Denied|DENIED|denied)

(Error|ERROR|error)

Sorting out (and finding!) connection resets on Linux:

Sorting out connection resets:

tcpdump -nn -v ‘tcp[tcpflags] & (tcp-rst) != 0’

or

tcpdump -ilo -nn -v ‘tcp[tcpflags] & (tcp-rst) != 0’

-n is to stop resolving DNS names

-nn is to stop resolving DNS names AND port numbers to names

This is really useful when an app is logging connection resets, but not telling you what it is trying to connect to in the first place.

Reading tags from CLI or bash in ec2 instances

# To read the Name tag:

TAG_NAME=”Name”

INSTANCE_ID=”`./ec2-metadata -i | cut -f 2 -d ” ” `”

REGION=”`././ec2-metadata -z | cut -f 2 -d ” ” `”

REGION=${REGION%?}

TAG_VALUE=”`aws ec2 describe-tags –filters “Name=resource-id,Values=$INSTANCE_ID” “Name=key,Values=$TAG_NAME” –region $REGION –output=text | cut -f5`”

Gives:

set |grep TAG_VALUE

TAG_VALUE=’brad test box’

# To see ALL tags to see WTF: (use region from above)

aws ec2 describe-tags –region us-east-1 –output=text

# To read the Created-by custom tag:

TAG_NAME=”created-by”

INSTANCE_ID=”`./ec2-metadata -i | cut -f 2 -d ” ” `”

REGION=”`././ec2-metadata -z | cut -f 2 -d ” ” `”

REGION=${REGION%?}

TAG_VALUE=”`aws ec2 describe-tags –filters “Name=resource-id,Values=$INSTANCE_ID” “Name=key,Values=$TAG_NAME” –region $REGION –output=text | cut -f5`”

set |grep TAG_VALUE

TAG_VALUE=test-kitchen

# To read the nifi-node-num custom tag:

TAG_NAME=”nifi-node-num”

echo “TAG_NAME is: ” ${TAG_NAME}

INSTANCE_ID=”`./ec2-metadata -i | cut -f 2 -d ” ” `”

echo “INSTANCE_ID is: ” ${INSTANCE_ID}

REGION=”`././ec2-metadata -z | cut -f 2 -d ” ” `”

REGION=${REGION%?}

echo “REGION is: ” ${REGION}

TAG_VALUE=”`aws ec2 describe-tags –filters “Name=resource-id,Values=$INSTANCE_ID” “Name=key,Values=$TAG_NAME” –region $REGION –output=text | cut -f5`”

echo “TAG_VALUE is: ” ${TAG_VALUE}

# You need the EC2 Metadata binary for the above to work

wget http://s3.amazonaws.com/ec2metadata/ec2-metadata

chmod u+x ec2-metadata

./ec2-metadata –help

You can also do this IF you have credentials:

aws ec2 describe-instances –region us-east-1 –instance-id i-09301dcede4431741|grep -A 200 Tag

                    “Tags”: [

                        {

                            “Value”: “False”,

                            “Key”: “data_sensitive”

                        },

                        {

                            “Value”: “10/31/2017”,

                            “Key”: “valid_thru”

                        },

                        {

                            “Value”: “nifi-al”,

                            “Key”: “Application”

                        },

                        {

                            “Value”: “Aplha”,

                            “Key”: “Cluster”

                        },

                        {

                            “Value”: “False”,

                            “Key”: “Docker”

                        },

                        {

                            “Value”: “TBD”,

                            “Key”: “BAPP_ID”

                        },

                        {

                            “Value”: “nifi-al-latest”,

                            “Key”: “Name”

                        },

                        {

                            “Value”: “WDPRTechnologyIAParksDataPlatform@disney.com“,

                            “Key”: “Owner”

                        },

                        {

                            “Value”: “1”,

                            “Key”: “nifi-node-num”

                        },

                        {

                            “Value”: “nifi-al-latest-asg”,

                            “Key”: “aws:autoscaling:groupName”

                        },

                        {

                            “Value”: “Non-Prod Sandbox”,

                            “Key”: “Environment”

                        },

                        {

                            “Value”: “yes”,

                            “Key”: “tag_compliance”

                        }

                    ],

                    “AmiLaunchIndex”: 2

                }

            ],

            “ReservationId”: “r-0a402050d68688b53”,

            “RequesterId”: “226008221399”,

            “Groups”: [],

            “OwnerId”: “876496569223”

        }

    ]

}

ALL ec2-metadata options:

Usage: ec2-metadata <option>

Options:

–all                     Show all metadata information for this host (also default).

-a/–ami-id               The AMI ID used to launch this instance

-l/–ami-launch-index     The index of this instance in the reservation (per AMI).

-m/–ami-manifest-path    The manifest path of the AMI with which the instance was launched.

-n/–ancestor-ami-ids     The AMI IDs of any instances that were rebundled to create this AMI.

-b/–block-device-mapping Defines native device names to use when exposing virtual devices.

-i/–instance-id          The ID of this instance

-t/–instance-type        The type of instance to launch. For more information, see Instance Types.

-h/–local-hostname       The local hostname of the instance.

-o/–local-ipv4           Public IP address if launched with direct addressing; private IP address if launched with public addressing.

-k/–kernel-id            The ID of the kernel launched with this instance, if applicable.

-z/–availability-zone    The availability zone in which the instance launched. Same as placement

-c/–product-codes        Product codes associated with this instance.

-p/–public-hostname      The public hostname of the instance.

-v/–public-ipv4          NATted public IP Address

-u/–public-keys          Public keys. Only available if supplied at instance launch time

-r/–ramdisk-id           The ID of the RAM disk launched with this instance, if applicable.

-e/–reservation-id       ID of the reservation.

-s/–security-groups      Names of the security groups the instance is launched in. Only available if supplied at instance launch time

-d/–user-data            User-supplied data.Only available if supplied at instance launch time.

If you don’t have credentials, some of this doesn’t work, but you can try this:

Websphere system core dump location is incorrect by default

Location of the system core dump is typically incorrect and just drops it in the middle of the host OS, you will need to edit the

/opt/apps/WebSphere/AppServer/bin/setupCmdLine.sh

# Add:

IBM_COREDIR=/net/cn-flor-nas01-prod.wdw.disney.com/data/TPR/WDW/Vol005/WDPRTHome/middleware/dumps/${HOSTNAME}

export COREDIR

# the COREDIR can also go at the end of the large export command at the end of the script.

JVM restart(s) are required to pick up the change. Even though the system dump is initiated, you have to pick the JVM that initiates it, so it is related to the JVM. A full restart of all the JVM’s is technically needed, as is changing this file on every node in the cluster.

“Incorrect” in that the dump files are freaking gigantic and run the box out of space pretty quickly. Redirect these to the NAS.

git 2.19.1 upgrade

This should get handled as part of normal patching, but RHEL ships with a 1.8.x branch of Git. We aren’t cloning from public repos for most things, but I can’t be 100% about it, and workstations are probably vulnerable as well. We should all update our workstations, but it doesn’t appear to be that simple for Linux servers:

 

(2.19.1 is the version to upgrade to.)

 

Mac: brew upgrade git didn’t seem to update the cli git, still on 1.8.3.1, have to download and install from https://sourceforge.net/projects/git-osx-installer/files/git-2.19.0-intel-universal-mavericks.dmg/download?use_mirror=autoselect

 

Windows: https://github.com/git-for-windows/git/releases/download/v2.19.1.windows.1/Git-2.19.1-64-bit.exe

 

Linux: Looks like git does not have an rpm for a 2.x, it’s a clone and compile, which makes long term sustainability a pain.

CVE says version 2.18 is the oldest affected version, and the newest available from the yum repos is 1.8.3.1-14.

https://git-scm.com/download/linux says to download and compile.

 

This works:

#!/bin/bash

# update git to 2.19.1

git –version

which git

cp /usr/bin/git /usr/bin/git-1.8.3.1

yum -y install curl-devel expat-devel gettext-devel openssl-devel zlib-devel

yum -y install gcc perl-ExtUtils-MakeMaker

cd /usr/src

wget https://www.kernel.org/pub/software/scm/git/git-2.19.1.tar.gz

tar xzf git-2.19.1.tar.gz

cd git-2.19.1

make prefix=/usr/local/git all

make prefix=/usr/local/git install

rm -f /bin/git

ln -s /usr/local/git/bin/git /bin/git

git –version

 

chef-client fails with ERROR: The used Encrypted Data Bags version requires an OpenSSL version with “aes-256-gcm” algorithm support

TL;DR:

Your path is probably wrong for the root user that is running chef-client. We had a long screwed up path that eventually included the right path, but had an old ChefDK preceeding it. Basically, you’re using a broken chef-client (too  new, too old, broken encryption, whatever)

Works:

PATH=/usr/bin:/bin:/etc:.:/usr/local/bin:/usr/sbin:/opt/OV/bin/OpC:/home/rundeck/:/opt/middleware/Tools

Does not: 

PATH=/usr/local/rvm/gems/ruby-2.3.1/bin:/usr/local/rvm/gems/ruby-2.3.1@global/bin:/usr/local/rvm/rubies/ruby-2.3.1/bin:/usr/kerberos/sbin:/usr/kerberos/bin:/root:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/rvm/bin:/root/bin

Fix:

export PATH=/usr/bin:/bin:/etc:.:/usr/local/bin:/usr/sbin:/opt/OV/bin/OpC:/home/rundeck/:/opt/middleware/Tools

Permanent fix:

vi /etc/bash.bashrc

Change to:

type rvm >/dev/null 2>/dev/null || echo ${PATH} | __rvm_grep “/usr/local/rvm/bin” > /dev/null || export PATH=”${PATH}:/usr/local/rvm/bin”

vi /root/.bashrc

export PATH=/usr/bin:$PATH:/root/bin

But this hoses up ruby, better to get the right version of chef-client:

mv /usr/local/rvm/gems/ruby-2.3.1/bin/chef-client /usr/local/rvm/gems/ruby-2.3.1/bin/chef-client-12.21.1

mv /usr/local/rvm/gems/ruby-2.3.1@global/bin/chef-client /usr/local/rvm/gems/ruby-2.3.1@global/bin/chef-client-12.21.1

mv /usr/local/rvm/rubies/ruby-2.3.1/bin/chef-client /usr/local/rvm/rubies/ruby-2.3.1/bin/chef-client-12.21.1

/usr/bin/chef-client –version

cd /usr/local/rvm/gems/ruby-2.3.1/bin/

ln /usr/bin/chef-client chef-client

chef-client –version

If you are running a specific version of chef-client, but running bare chef-client gives the wrong version, check “which chef-client” and rename the ones buried in the ruby paths:

mv /usr/local/rvm/gems/ruby-2.5.3/bin/chef-client /usr/local/rvm/gems/ruby-2.5.3/bin/chef-client.12.19.36

mv /usr/local/rvm/gems/ruby-2.4.2/bin/chef-client /usr/local/rvm/gems/ruby-2.4.2/bin/chef-client.14.9.13

etc.

logout and back in

chef-client -v 

should give you the proper version.

Frequent failure modes:

Needs newer version 12.21.31+:

[2019-03-18T14:50:54+00:00] ERROR: Cookbook ‘wdpr_dnsmasq’ version ‘0.0.1’ depends on chef version [“>= 12.21.31”], but the running chef version is 12.19.36

[2019-03-18T14:50:54+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

Needs older version until cook book is updated:

Indicative of running chef-client 14 against incompatible cookbooks:

Running handlers:

[2019-03-18T14:54:29+00:00] ERROR: Running exception handlers

Running handlers complete

[2019-03-18T14:54:29+00:00] ERROR: Exception handlers complete

Chef Client failed. 0 resources updated in 07 seconds

[2019-03-18T14:54:30+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out

[2019-03-18T14:54:30+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report

[2019-03-18T14:54:30+00:00] ERROR: can’t modify frozen Array

[2019-03-18T14:54:30+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

590  [2018-10-10 15:27:33] echo ${PATH}

  591  [2018-10-10 15:29:27] chef-client

  592  [2018-10-10 15:30:07] which chef-client

  593  [2018-10-10 15:30:28] /usr/local/rvm/gems/ruby-2.3.1/bin/chef-client –version

  594  [2018-10-10 15:30:40] mv /usr/local/rvm/gems/ruby-2.3.1/bin/chef-client /usr/local/rvm/gems/ruby-2.3.1/bin/chef-client-12.21.1

  595  [2018-10-10 15:30:43] which chef-client

  596  [2018-10-10 15:30:55] /usr/local/rvm/gems/ruby-2.3.1@global/bin/chef-client –version

  597  [2018-10-10 15:31:08] mv /usr/local/rvm/gems/ruby-2.3.1@global/bin/chef-client /usr/local/rvm/gems/ruby-2.3.1@global/bin/chef-client-12.21.1

  598  [2018-10-10 15:31:10] which chef-client

  599  [2018-10-10 15:31:20] /usr/local/rvm/rubies/ruby-2.3.1/bin/chef-client –version

  600  [2018-10-10 15:31:55] mv /usr/local/rvm/rubies/ruby-2.3.1/bin/chef-client /usr/local/rvm/rubies/ruby-2.3.1/bin/chef-client-12.21.1

  601  [2018-10-10 15:31:57] which chef-client

  602  [2018-10-10 15:32:05] /usr/bin/chef-client –version

  603  [2018-10-10 15:32:15] chef-client

  604  [2018-10-10 15:32:24] which chef-client

  605  [2018-10-10 15:32:30] chef-client

  606  [2018-10-10 15:32:38] cd /usr/local/rvm/gems/ruby-2.3.1/bin/

  607  [2018-10-10 15:32:39] ll

  608  [2018-10-10 15:33:28] ln chef-client /usr/bin/chef-client

  609  [2018-10-10 15:33:38] ln /usr/bin/chef-client chef-client

  610  [2018-10-10 15:33:42] chef-client

  611  [2018-10-10 15:35:00] history

[rundeck][nl-fldi-02119][~]

$ chef-client –version

Chef: 12.19.36

Encrypted Data Bags version requires an OpenSSL version with “aes-256-gcm” algorithm support

openssl enc -help 2>&1 | grep gcm

[bwilliam@nl-fldi-02119 ~]$ openssl enc -help 2>&1 | grep gcm

-aes-128-ctr               -aes-128-ecb               -aes-128-gcm

-aes-192-gcm               -aes-192-ofb               -aes-256-cbc

-aes-256-ecb               -aes-256-gcm               -aes-256-ofb

The used Encrypted Data Bags version requires an OpenSSL version with “aes-256-gcm” algorithm support

knife data bag show −−secret-file=./rev_secret_key rev_secret revpass

knife data bag show −−secret-file=/etc/chef/encrypted_data_bag_secret users rundeck

[2018-10-10T09:38:11-04:00] ERROR: The used Encrypted Data Bags version requires an OpenSSL version with “aes-256-gcm” algorithm support

Recipe Compile Error in /var/chef/cache/cookbooks/wdprt_rundeck_client/recipes/default.rb

  ================================================================================

  Chef::EncryptedDataBagItem::EncryptedDataBagRequirementsFailure

  —————————————————————

  The used Encrypted Data Bags version requires an OpenSSL version with “aes-256-gcm” algorithm support

[root@nl-fldi-02119 chef]# chef-client –version

Chef: 12.21.1

knife data bag show users rundeck

  cipher:         aes-256-gcm

which knife

which chef-client

[rundeck][nl-fldi-02119][~]

$ chef-client –version

Chef: 12.19.36

knife data bag show users rundeck