Ansible Galaxy error “list index out of range” during role installation

ERROR! Unexpected Exception, this is probably a bug: list index out of range

was the message I was met with when recently tried using ansible-galaxy.  I decided to use Ansible for a Jenkins project instead of Puppet because a role for Jenkins v2 already exists on Ansible Galaxy.

The error

When I attempted to install the Jenkins role using the ansible-galaxy command into my home directory I received what looks like a programming error.

$ ansible-galaxy install geerlingguy.jenkins
- downloading role 'jenkins', owned by geerlingguy
- downloading role from https://github.com/geerlingguy/ansible-role-jenkins/archive/3.2.2.tar.gz
- extracting geerlingguy.jenkins to /etc/ansible/roles/geerlingguy.jenkins
- extracting geerlingguy.jenkins to /usr/share/ansible/roles/geerlingguy.jenkins
ERROR! Unexpected Exception, this is probably a bug: list index out of range
to see the full traceback, use -vvv

list index out of range usually means something internal failed in a bad way.

The solution

Turning to Google, I found https://github.com/ansible/galaxy/issues/149.   Basically, ansible-galaxy does not take into which account is running the ansible-galaxy command.  It reads the ansible configuration file and uses the defined directory.  This results in ansible-galaxy wanting to write to /etc/ansible which a regular user account cannot write to.

Using the -p flag I was able to my home directory and install the role.

$ ansible-galaxy install -vvv -p /home/example/src/ansible/jenkins/roles geerlingguy.jenkins
ansible-galaxy 2.4.1.0
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/example/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible-galaxy
python version = 2.7.5 (default, Aug 4 2017, 00:39:18) [GCC 4.8.5 20150623 (Red Hat 4.8.5-16)]
Using /etc/ansible/ansible.cfg as config file
Opened /home/example/.ansible_galaxy
Processing role geerlingguy.jenkins
Opened /home/example/.ansible_galaxy
- downloading role 'jenkins', owned by geerlingguy
https://galaxy.ansible.com/api/v1/roles/?owner__username=geerlingguy&name=jenkins
https://galaxy.ansible.com/api/v1/roles/440/versions/?page_size=50
- downloading role from https://github.com/geerlingguy/ansible-role-jenkins/archive/3.2.2.tar.gz
- extracting geerlingguy.jenkins to /home/example/src/ansible/jenkins/roles/geerlingguy.jenkins
- geerlingguy.jenkins (3.2.2) was installed successfully
- adding dependency: geerlingguy.java
Processing role geerlingguy.java
Opened /home/example/.ansible_galaxy
- downloading role 'java', owned by geerlingguy
https://galaxy.ansible.com/api/v1/roles/?owner__username=geerlingguy&name=java
https://galaxy.ansible.com/api/v1/roles/439/versions/?page_size=50
- downloading role from https://github.com/geerlingguy/ansible-role-java/archive/1.7.4.tar.gz
- extracting geerlingguy.java to /home/example/src/ansible/jenkins/roles/geerlingguy.java
- geerlingguy.java (1.7.4) was installed successfully

 

Securing /etc/puppet on CentOS

I’ve recently delved in to the world of Puppet to manage some CentOS servers. In the process I noticed something. The /etc/puppet directory is owned by root:root but puppet runs as the user puppet. What does this mean? A couple of things:

  • To edit the manifests or modules I either have to be root or constantly be typing sudo (annoying).
  • For the puppetmaster process, which runs as puppet:puppet to access the files, the manifest and modules must be world readable.  This means a lot of information is visible to the world, encrypted or not.
  • I can’t use my favorite editor to edit files over ssh. (I know, a personal gripe, but valid in my books.)

So I’m trying an experiment that I hope will secure the data a bit more and make editing the files more hastle free.

  • Recursively changed the group of /etc/puppet to puppet.
  • Put myself in the puppet group.  I can now edit the files without being root.  (See newgrp(1).)
  • I’ll slowly begin to set the Other permission bits to 0, hiding the files and their contents from prying eyes.

 

 

 

Puppet-lint Plugins List

Based on the Puppet-lint Plugins list available at the Puppet Community site, I’ve added the gem command line and Gemfile commands for easy installation.

absolute_classname

  • Check relative class name inclusions.
  • gem install puppet-lint-absolute_classname-check

absolute_template

  • Check if paths to the template() function are relative.
  • gem install puppet-lint-absolute_template_path

alias

  • Check for alias parameters in resources.
  • gem install puppet-lint-alias-check
  • gem 'puppet-lint-alias-check', :require => false

appends

  • Check that the appends operator (+=) is not used (removed in Puppet 4.0.0).
  • gem install puppet-lint-appends-check
  • gem 'puppet-lint-appends-check', :require => false

classes_and_types_beginning_with_digits

  • Check for types and class names that begin with digits.
  • gem install puppet-lint-classes_and_types_beginning_with_digits-check
  • gem 'puppet-lint-classes_and_types_beginning_with_digits-check', :require => false

empty_string

  • Check for variables assigned to the empty string.
  • gem install puppet-lint-classes_and_types_beginning_with_digits-check
  • gem 'puppet-lint-empty_string-check', :require => false

file_ensure

  • Check the ensure attribute on file resources.
  • gem install puppet-lint-file_ensure-check
  • gem 'puppet-lint-file_ensure-check', :require => false

file_source_rights

  • Check file rights when providing a source.
  • gem install puppet-lint-file_source_rights-check
  • gem 'puppet-lint-file_source_rights-check', :require => false

fileserver

  • Check if puppet:/// is used instead of file().
  • gem install puppet-lint-fileserver-check

global_resource

  • Ensure that your manifests have no global resources.
  • gem install puppet-lint-global_resource-check

leading_zero

  • Check for unquoted numbers with leading zero.
  • gem install puppet-lint-leading_zero-check
  • gem 'puppet-lint-leading_zero-check', :require =>; false

newmericvariable

  • Extends puppet-lint to ensure that your variables are not numeric.
  • gem install puppet-lint-numericvariable

package_ensure

  • Check the ensure attribute on package resources.
  • gem install puppet-lint-package_ensure-check
  • gem 'puppet-lint-package_ensure-check'

param_docs

  • Check that validates all parameters are documented.
  • gem install puppet-lint-param-docs

resource_outside_class

  • Check if resources exist outside of a class or defined type.
  • gem install puppet-lint-resource_outside_class-check

resource_reference_syntax

  • Ensure that the reference syntax follows Puppet 4 style.
  • gem install puppet-lint-resource_reference_syntax
  • gem 'puppet-lint-resource_reference_syntax'

roles_and_profiles

  • Check that a node definition declares only a role, a role class does not have any param and only declares profiles, and a profiles class can declare anything but a role. gem install puppet-lint-roles_and_profiles-check
  • gem install puppet-lint-roles_and_profiles-check
  • gem 'puppet-lint-roles_and_profiles-check'

security

  • Checks puppet manifests for security related problems.
  • gem install puppet-lint-security-plugins

spaceship_operator_without_tag

  • Check that spaceship operator is called with a tag.
  • gem install puppet-lint-spaceship_operator_without_tag-check
  • gem 'puppet-lint-spaceship_operator_without_tag-check', :require => false

strict_indent

  • Ensure that your manifests follow a strict indentation pattern.
  • gem install puppet-lint-strict_indent-check

trailing_comma

  • Check for missing trailing commas.
  • gem install puppet-lint-trailing_comma-check
  • gem 'puppet-lint-trailing_comma-check', :require => false

trailing_newline

  • Ensure that your manifest files end with newlines.
  • gem install puppet-lint-trailing_newline-check

undef_in_function

  • Check for undef in function calls.
  • gem install puppet-lint-undef_in_function-check

unquoted_string

  • Check that selectors and case statements cases are quoted.
  • gem install puppet-lint-unquoted_string-check
  • gem 'puppet-lint-unquoted_string-check', :require => false

usascii_format

  • Check that manifest files contain only US ASCII.
  • gem install puppet-lint-usascii_format-check

variable_contains_upcase

  • Ensure that your variables are all lower case.
  • gem install puppet-lint-variable_contains_upcase

version_comparison

  • Check for versions compared as numbers.
  • gem install puppet-lint-version_comparison-check
  • gem 'puppet-lint-version_comparison-check', :require => false

vim_modeline

  • Check for vim comment (modeline) as the last line in a manifest.
  • gem install puppet-lint-vim_modeline-check

Installing puppet-rspec on CentOS 6

Here are the steps I used to install puppet-rspec on CentOS 6.

I’ll be using RVM to install a recent version of Ruby to ease the process.  At one point you could get rpec to work with the Ruby 1.8.7, the version that ships with CentOS 6, but recent attempts have failed for me because the required GEMs all require newer Ruby versions.

Installing RVM

1) Install RVM.  We’ll be using the multi-user install, but the single user install works also.

sudo yum install gnupg2
sudo gpg --keyserver hkp://keys.gnupg.net --recv-keys 409B6B1796C275462A1703113804BB82D39DC0E3
curl -sSL https://get.rvm.io | sudo bash -s stable

RVM installs a file in /etc/profile.d that allows it to take over ruby duties when anyone logs in. If you prefer to selectively enable RVM, move the /etc/profile.d/rvm.sh file to another location and run the command
“source /path/to/rvm.sh” to enable it.

2) Add your user account to the rvm group to give yourself the ability to use rvm to manage ruby.  (You cannot use root or sudo to manage rvm.)

sudo usermod -a -G rvm myuser

Restart your session to pick up the change.

3) Install a newer Ruby.

rvm install 2.1

Run ruby --version to make sure you’re using the installed version. You should see something similar to:

ruby 2.1.5p273 (2014-11-13 revision 48405) [x86_64-linux]

Installing puppet-rspec

4) Install the puppet-rspec gem along with some other requirements.

You’ll notice I have puppet in the list of gems to install.  Now that we are using RVM,  the Puppet gem installed at the system level is no longer visible, so we need to reinstall it.   Make sure to specify the version if you don’t want to run the latest release.  Here I install version 3.8.1, the latest version 3 at this time.

gem update --system
gem install puppet -v 3.8.1
gem install -V rspec
gem install -V rspec-puppet
gem install -V puppet-lint
gem install -V puppet-syntax
# No need to install these two, they get pulled in by rspec.
#gem install -V rspec-core
#gem install -V rspec-expectations

5) Build and install the puppetlabs-rspec-helper gem.  We’ll use puppetlabs-rspec-helper in the next step to setup our directories.

git clone git://github.com/puppetlabs/puppetlabs_spec_helper.git
cd puppetlabs_spec_helper
rake package:gem
gem install -V pkg/puppetlabs_spec_helper-*.gem

Configuring the module

6) Run puppetlabs-rspec-helper to initialize the module’s spec directory layout.

cd path/to/your/module
rspec-puppet-init

 

7) Add the following line to your module’s spec/spec_helper.rb file.

require 'puppetlabs_spec_helper/module_spec_helper'

 

8) Add the following line to your module’s Rakefile:

require 'puppetlabs_spec_helper/rake_tasks'

 

9) Create a puppetlabs_spec_helper .fixtures.yml file in your module’s directory. Here is an example file:

fixtures:
  repositories:
    stdlib:
      repo: 'git://github.com/puppetlabs/puppetlabs-stdlib'
      ref: '4.6.0'
    symlinks:
      'atd': "#{source_dir}"

 

If you don’t create the .fixtures.yml you will receive trace backs.  In my case rake complained about missing constants.

So that that’s it.  Hopefully you found this useful and it got you up and running.

 

Useful Links

https://rvm.io/rvm/install
https://github.com/puppetlabs/puppetlabs_spec_helper (explains .fixures.yml)
http://rspec-puppet.com/
https://github.com/rodjek/rspec-puppet
https://puppetlabs.com/blog/the-next-generation-of-puppet-module-testing