Category Archives: linux

All things linux related (Mostly Redhat)

Open vSwitch: basic initial setup on Ubuntu 12.04

Software defined networking is all the rage these days in IT, and Open vSwitch is where most people get their first introduction. Whether you’re running KVM or Xen, Open vSwitch will allow you to abstract network setup functions from the hypervisor. Here’s a quick guide to setting up Open vSwitch and replacing a physical interface with a virtual one:
Continue reading

haproxy + keepalived – the free HA load balancer

Load balancers are cool, especially free ones. haproxy and keepalived together can give you a simple HA load balancer at the cost of the hardware you run it on. Here’s how to setup a basic active/passive load balancer with haproxy and keepalived. First the environment:
Continue reading

Galera / WSREP – MySQL cluster on Ubuntu 12.04 LTS – basic initial setup

mysql-logoIt took me about 4 hours of actual research, and trial and error, however I finally got a working Galera based WSREP MySQL cluster up and running. Here’s what it took and how the pieces came together:

Getting the install right took a couple of hours by itself for some reason, here’s the order that worked best on Ubuntu 12.04 LTS:
Continue reading

OpenStack API python script example

Recently I was given a number of API test cases for OpenStack to manually execute via curl. It was quite tedious figuring out the nuances of the OpenStack API using curl; manually constructing HTTP headers and substituting JSON values into a curl command got old real fast. I decided to run all the test cases through python, this was the first bit I had to finish, authentication!

OpenStack API call requesting a token. Tokens are valid for authentication for 24 hours:

#!/usr/bin/python

import argparse
import getopt
import json
import sys
import urllib2

def getToken(url, osuser, ostenant, ospassword):

    """
    Returns a token to the user given a tenant,
    user name, password, and OpenStack API URL.
    """

    url = url + '/tokens'
    tokenRequest = urllib2.Request(url)
    tokenRequest.add_header("Content-type", "application/json")
    jsonPayload = json.dumps({'auth' : {'tenantName' : ostenant, 'passwordCredentials' : {'username' : osuser, 'password' : ospassword}}})
   
    request = urllib2.urlopen(tokenRequest, jsonPayload)
    json_data = json.loads(request.read())
   
    request.close()
    return json.dumps(json_data)

# Build our required arguments list
parser = argparse.ArgumentParser()
mandatory = parser.add_argument_group("mandatory")
mandatory.add_argument("-l", "--login", help="The administrative user for your OpenStack installation", type=str)
mandatory.add_argument("-p", "--password", help="The administrative user's password", type=str)
mandatory.add_argument("-t", "--tenant", help="The administrative user's tenant / project", type=str)
mandatory.add_argument("-u", "--url", help="The Keystone API endpoint from running, 'nova endpoints'", type=str)
args = parser.parse_args()

# Validate arugments were given
if type(args.url) != type(str()):
    sys.stderr.write('Invalid URL: %s\n' % args.url)
    parser.print_help()
    sys.exit(2)
if type(args.tenant) != type(str()):
    sys.stderr.write('Invalid tenant: %s\n' % args.tenant)
    parser.print_help()
    sys.exit(2)
if type(args.password) != type(str()):
    sys.stderr.write('Invalid password: %s\n' % args.password)
    parser.print_help()
    sys.exit(2)
if type(args.login) != type(str()):
    sys.stderr.write('Invalid login: %s\n' % args.login)
    parser.print_help()
    sys.exit(2)
   
# Since we return a raw JSON payload from getToken,
# we need to load it into a readable object.
adminToken = json.loads(getToken(args.url, args.login, args.tenant, args.password))

# Access the token portion of the JSON payload and grab the token and tenant ID
adminTokenID = adminToken['access']['token']['id']
adminTokenTenantID = adminToken['access']['token']['tenant']['id']

for item in adminToken['access']['serviceCatalog']:
    """
    The "name" of each OpenStack service catalog item from
    the item list changed between versions.  Things like
    "glance" became "volume" and "keystone" became "identity".  
    You will need to update this based on your installation.
    """

    if item['name'] == "nova":
        adminNovaURL = item['endpoints'][0]['adminURL']
    if item['name'] == "glance":
        adminGlanceURL = item['endpoints'][0]['adminURL']
    if item['name'] == "nova-volume":
        adminVolumeURL = item['endpoints'][0]['adminURL']
    if item['name'] == "ec2":
        adminEc2URL = item['endpoints'][0]['adminURL']
    if item['name'] == "swift":
        adminSwiftURL = item['endpoints'][0]['adminURL']
    if item['name'] == "keystone":
        adminAuthURL = item['endpoints'][0]['adminURL']

print "------ Admin information ------"
print "Token ID = ", adminTokenID
print "Tenant ID = ", adminTokenTenantID
print "Nova URL = ", adminNovaURL
print "Glance URL = ", adminGlanceURL
print "Volume URL = ", adminVolumeURL
print "EC2 URL = ", adminEc2URL
print "Swift URL = ", adminSwiftURL
print "Keystone URL = ", adminAuthURL
print "------ End Admin info ---------"

When passed the appropriate parameters, this script will print your environment information in a nice readable format:

uesr@localhost:~$ ./token.py -l admin -t admin -p password -u http://localhost:35357/v2.0
------ Admin information ------
Token ID =  abc123def456ghi789jkl012mno345pq
Tenant ID =  qp543onm210lkj987ihg654fed321cba
Nova URL =  http://localhost:8774/v2/qp543onm210lkj987ihg654fed321cba
Glance URL =  http://localhost:9292/v1
Volume URL =  http://localhost:8776/v1/qp543onm210lkj987ihg654fed321cba
EC2 URL =  http://localhost:8773/services/Admin
Swift URL =  https://localhost:8080/v1/
Keystone URL =  http://localhost:35357/v2.0
------ End Admin info ---------

Hack the planet automobile?

I’ve been complaining to the wife that her car’s infotainment system is behind the curve a bit. This is an attempt to help spread the word about Automotive Grade Linux (AGL), and perhaps aid the open-source automotive community a bit. Here’s an excerpt from an article posted on wired:

The major roadblock to the rapid adoption of technology in cars has always been vehicles’ long production cycles. But automakers are also guilty of hampering wide-scale innovation with their proprietary approach to infotainment systems and software, while dragging their feet on a collaborative, open source solution that would benefit consumers and car tech overall.

Read the full post here.

/etc/security/limits.conf nofile absolute maximum

Apparently unlimited != unlimited in the Linux kernel for maximum number of open files. After some extensive digging, I finally found the actual maximum to the nofile setting in /etc/security/limits.conf. Yes I was searching in the context of Oracle (Imagine that) for a maximum number of procs / files. The Linux kernel has a hard upper limit of 1024*1024 (1048576 – a magical number I won’t soon forget).

Consider the following:

[root@localhost ~]# grep mrsmith /etc/security/limits.conf
mrsmith     soft    nofile      unlimited  
mrsmith     hard    nofile      unlimited

Trying to su to mrsmith suddenly casts you aside:

[root@localhost ~]# su - mrsmith
could not open session

Displaying the open file setting for mrsmith shows something odd:

[root@localhost ~]# ulimit -au mrsmith
...
open files                      (-n) 1024

Why would our number of open files be 1024? This apparently is the default setting for users without a custom nofile set based on the following bugzilla.

Then what is our actual upper bound for nofile in /etc/security/limits.conf? This isn’t well documented anywhere I’ve seen, but based on some research it appears to be 1024*1024. Setting nofile to 1048577 produces the same “could not open session” error as mentioned previously. 1048576 however seems to work just fine. Although you can set your nofile ulimit to this, the effects of such are unknown and I would highly recommend testing any such setting.

kvm-qemu: scripted clones of virtual machines

KVM is quickly becoming a viable competitor to paid vendors given the wide variety of features it offers in comparison (Memory page sharing, live migration, HA clustering, the list goes on and on…). Cloning virtual machines is common practice when deploying large environments with many virtual machines. Here’s a quick and dirty way to clone a virtual machine that could easily be scripted to deploy a large number of virtual machines:
Continue reading