TripleO Container steps

Container steps

Similar to bare metal, containers are brought up in a stepwise manner. The current architecture supports bringing up baremetal services alongside containers. Therefore, baremetal steps may be required depending on the service and they are always executed before the corresponding container step.

The list below represents the correlation between the baremetal and the container steps. These steps are executed sequentially:

• Containers config files generated per hiera settings.

• Host Prep

• Load Balancer configuration baremetal

  • Step 1 external steps (execute Ansible on Undercloud)

  • Step 1 deployment steps (Ansible)

  • Common Deployment steps

    • Step 1 baremetal (Puppet)

    • Step 1 containers

• Core Services (Database/Rabbit/NTP/etc.)

  • Step 2 external steps (execute Ansible on Undercloud)

  • Step 2 deployment steps (Ansible)

  • Common Deployment steps

    • Step 2 baremetal (Puppet)

    • Step 2 containers

• Early Openstack Service setup (Ringbuilder, etc.)

  • Step 3 external steps (execute Ansible on Undercloud)

  • Step 3 deployment steps (Ansible)

  • Common Deployment steps

    • Step 3 baremetal (Puppet)

    • Step 3 containers

• General OpenStack Services

  • Step 4 external steps (execute Ansible on Undercloud)

  • Step 4 deployment steps (Ansible)

  • Common Deployment steps

    • Step 4 baremetal (Puppet)

    • Step 4 containers (Keystone initialization occurs here)

• Service activation (Pacemaker)

  • Step 5 external steps (execute Ansible on Undercloud)

  • Step 5 deployment steps (Ansible)

  • Common Deployment steps

    • Step 5 baremetal (Puppet)

    • Step 5 containers

View the list of images on the undercloud docker-distribution registry

To view the list of images on the undercloud docker-distribution registry use the following command:

(undercloud) $ curl http://192.168.24.1:8787/v2/_catalog | jq .repositories[]

To view a list of tags for a specific image, use the skopeo command:

(undercloud) $ curl -s http://192.168.24.1:8787/v2/rhosp13/openstack-keystone/tags/list | jq .tags

To verify a tagged image, use the skopeo command:

(undercloud) $ skopeo inspect --tls-verify=false docker://192.168.24.1:8787/rhosp13/openstack-keystone:13.0-44

Updating network configuration on the Overcloud after a deployment

By default, subsequent change(s) made to network configuration templates (bonding options, mtu, bond type, etc) are not applied on existing nodes when the overcloud stack is updated.

To push an updated network configuration add UPDATE to the list of actions set in the NetworkDeploymentActions parameter. (The default is ['CREATE'], to enable network configuration on stack update it must be changed to: ['CREATE','UPDATE'].)

• Enable update of the network configuration for all roles by adding the following to parameter_defaults in an environment file:

  parameter_defaults:
    NetworkDeploymentActions: ['CREATE','UPDATE']

• Limit the network configuration update to nodes of a specific role by using a role-specific parameter, i.e: {role.name}NetworkDeploymentActions. For example to update the network configuration on the nodes in the Compute role, add the following to parameter_defaults in an environment file:

  parameter_defaults:
    ComputeNetworkDeploymentActions: ['CREATE','UPDATE']

OSD refusing to start with "ERROR: osd init failed: (1) Operation not permitted"

The main issue is: OSD refuses to start with "ERROR: osd init failed: (1) Operation not permitted"

Log error:

2014-11-13 02:32:32.380964 7f977fd87780 1 journal _open /var/lib/ceph/osd/ceph-289/journal fd 21: 10736369664 bytes, block size 4096 bytes, directio = 1, aio = 1
2014-11-13 02:32:32.393814 7f977fd87780 1 journal _open /var/lib/ceph/osd/ceph-289/journal fd 21: 10736369664 bytes, block size 4096 bytes, directio = 1, aio = 1
2014-11-13 02:32:42.105930 7f977fd87780 1 journal close /var/lib/ceph/osd/ceph-289/journal
2014-11-13 02:32:42.112233 7f977fd87780 -1 ** ERROR: osd init failed: (1) Operation not permitted

Resolution:

• It appears the OSD is having trouble authenticating with the monitor.
• Verify the keyring file is present and correct?
• By default, it is located in /var/lib/ceph/osd/ceph-/keyring.
• It should match the key returned from the command

# ceph auth get osd.

Get IPMI IP address from OS

 First check that you have ipmitool installed:

[root@lykan ~]# yum provides ipmitool Last metadata expiration check: 0:06:54 ago on Thu 21 Nov 2019 10:39:22 PM CST. ipmitool-1.8.18-10.fc29.x86_64 : Utility for IPMI control Repo : fedora Matched from: Provide : ipmitool = 1.8.18-10.fc29

Discover:

[root@lykan ~]# ipmitool lan print | grep "IP Address" IP Address Source : Static Address IP Address : 10.10.4.5

The complete information provided:

[root@lykan ~]# ipmitool lan print Set in Progress : Set Complete Auth Type Support : NONE MD2 MD5 PASSWORD Auth Type Enable : Callback : : User : : Operator : : Admin : : OEM : IP Address Source : Static Address IP Address : 10.10.4.5 Subnet Mask : 255.255.255.0 MAC Address : xx:xx:xx:xx:xx:xx SNMP Community String : public IP Header : TTL=0x40 Flags=0x00 Precedence=0x00 TOS=0x10 BMC ARP Control : ARP Responses Disabled, Gratuitous ARP Disabled Gratituous ARP Intrvl : 2.0 seconds Default Gateway IP : 10.10.4.254 Default Gateway MAC : 00:00:00:00:00:00 Backup Gateway IP : 0.0.0.0 Backup Gateway MAC : 00:00:00:00:00:00 802.1q VLAN ID : Disabled 802.1q VLAN Priority : 0 RMCP+ Cipher Suites : 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,128 Cipher Suite Priv Max : XXXaaaXXaaaXaaa : X=Cipher Suite Unused : c=CALLBACK : u=USER : o=OPERATOR : a=ADMIN : O=OEM Bad Password Threshold : Not Available

Improve user experience using QEMU/KVM with Windows guest

A lot of sysadmins, SRE o wherever you want to call us, using native Linux in our laptops have the need to use virtual machines running Windows (some support, pentesting tasks, etc), if you are passionate about running periodic updates by now you figure out the main problem of this, if not, you will; the main problem is that on every kernel upgrade, you will lose the modules of VMware or VirtualBox, the best solution for this is to use QEMU/KVM, the K is for kernel so the support is embedded in the kernel, with this you will never lose support on your virtual machines, but there is a catch, even if you install virtIO drivers you will face issues like the screen does not resize, copy and paste from host to guest does not work, and is very sad to work that way.

So the solution: The SPICE project aims to provide a complete open-source solution for remote access to virtual machines in a seamless way so you can play videos, record audio, share USB devices, and share folders without complications.

SPICE could be divided into 4 different components: Protocol, Client, Server, and Guest. The protocol is the specification in the communication of the three other components; A client such as a remote viewer is responsible to send data and translate the data from the Virtual Machine (VM) so you can interact with it; The SPICE server is the library used by the hypervisor in order to share the VM under SPICE protocol; And finally, the Guest side is all the software that must be running in the VM in order to make SPICE fully functional, such as the QXL driver and SPICE VDAgent.

Just put in your virtual machine a channel spice and install the driver, the latest version could be found here.

fake_pxe as pm_type in RHOSP13 (TripleO + OpenStack Queens)

So, in RHOSP13 fake_pxe is being deprecated to change in RHOSP14 for manual management, the problem is that is just in between the migration, so there is no a clean way to use fake_pxe in RHOSP13. Another change is in the installation of undercloud, the option enabled_drivers is now DEPRECATED and changed by enabled_hardware_types. What now, in order to be able to use fake_pxe as a pm_type first install the undercloud without the options enabled_drivers, only use enabled_hardware_types, and add at the end manual-management, like this:

... #enabled_drivers=pxe_drac,pxe_ilo,pxe_ipmitool enabled_hardware_types=redfish,ipmi,idrac,ilo,manual-management ...

After that just install the undercloud using the common way.

[stack@director01 ~]$ openstack undercloud install ... ############################################################################# Undercloud install complete. The file containing this installation's passwords is at /home/stack/undercloud-passwords.conf. There is also a stackrc file at /home/stack/stackrc. These files are needed to interact with the OpenStack services, and should be secured. ############################################################################# [stack@director01 ~]$

Next, change manually the ironic.conf file located in /etc/ironic/ironic.conf to enable the DEPRECATED option enabled_drivers and add fake as a new driver.

enabled_drivers=pxe_drac,pxe_ilo,pxe_ipmitool,fake enabled_hardware_types=redfish,ipmi,idrac,ilo,manual-management

And restart ironic-conductor service:

(undercloud) [stack@director01 ~]$ sudo systemctl restart openstack-ironic-conductor

check the drivers:

(undercloud) [stack@director01 ]$ openstack baremetal driver list +---------------------+------------------------+ | Supported driver(s) | Active host(s) | +---------------------+------------------------+ | idrac | director01 | | ilo | director01 | | ipmi | director01 | | manual-management | director01 | | pxe_drac | director01 | | pxe_ilo | director01 | | pxe_ipmitool | director01 | | redfish | director01 | +---------------------+------------------------+

Now can we add a instackenv.json file.

(undercloud) [stack@director01 ~]$ cat instackenv-controller01.json { "nodes":[ { "mac":["controller1_mac"], "name":"nuc-controller01", "arch":"x86_64", "capabilities":"profile:control,node:controller-0,boot_option:local", "pm_type":"fake" } ] }

If you don't do this or try to use manual-management pm_type at this moment you will get an error similar to this one:

(undercloud) [stack@director01 ~]$ openstack overcloud node import ~/instackenv-controller01.json Started Mistral Workflow tripleo.baremetal.v1.register_or_update. Execution ID: 6ce7871c-d9d0-448e-9b46-78ced387fa48 Waiting for messages on queue 'tripleo' with no timeout. No valid host was found. Reason: No conductor service registered which supports driver fake. (HTTP 400) Exception registering nodes: No valid host was found. Reason: No conductor service registered which supports driver fake. (HTTP 400)

Import the new node definition to ironic and run introspection:

(undercloud) [stack@director01 ~]$ openstack overcloud node import ~/instackenv-compute01.json Started Mistral Workflow tripleo.baremetal.v1.register_or_update. Execution ID: 434cfe01-740d-4d58-b504-6f291ab12823 Waiting for messages on queue 'tripleo' with no timeout. 1 node(s) successfully moved to the "manageable" state. Successfully registered node UUID 62ce7d2c-03ae-4c6e-8c4a-13e817f26fa3 (undercloud) [stack@director01 ~]$ (undercloud) [stack@director01 ~]$ openstack baremetal introspection start --wait nuc-controller01 Waiting for introspection to finish... +------------------+-------+ | UUID | Error | +------------------+-------+ | nuc-controller01 | None | +------------------+-------+

But as I said, the fake driver is not going to be supported in RHOSP14 so version 13, is in the middle of the migration and we can introspect the node using fake driver, but we are not going to be able to install it, if we tried so, we will get an error like this one:

(undercloud) [stack@director01 ~]$ openstack action execution output show a637a01a-5f66-48a0-9e25-96700240c17e { "result": "Invalid node data: unknown pm_type (ironic driver to use): manual" }

So in order to solve this we need to change the driver type directly in the database, first, find the password in ironic.conf file

(undercloud) [stack@director01 ~]$ grep mysql /etc/ironic/ironic.conf #mysql_engine = InnoDB connection=mysql+pymysql://ironic:38315b04050cd6ad074ae75855f7c4367299b61a@192.168.10.9/ironic # set this to no value. Example: mysql_sql_mode= (string #mysql_sql_mode = TRADITIONAL #mysql_enable_ndb = false

Then look for the drivers configured.

MariaDB [ironic]> select name,driver from nodes; +------------------+--------+ | name | driver | +------------------+--------+ | nuc-controller01 | fake | | nuc-compute01 | fake | | nuc-compute02 | fake | +------------------+--------+ 3 rows in set (0.00 sec) MariaDB [ironic]> update nodes set driver = "manual-management" where name = "nuc-controller01"; Query OK, 1 row affected (0.01 sec) Rows matched: 1 Changed: 1 Warnings: 0 MariaDB [ironic]> update nodes set driver = "manual-management" where name = "nuc-compute01"; Query OK, 1 row affected (0.01 sec) Rows matched: 1 Changed: 1 Warnings: 0 MariaDB [ironic]> update nodes set driver = "manual-management" where name = "nuc-compute02"; Query OK, 1 row affected (0.01 sec) Rows matched: 1 Changed: 1 Warnings: 0 MariaDB [ironic]> select name,driver from nodes; +------------------+--------+ | name | driver | +------------------+--------+ | nuc-controller01 | manual-management | | nuc-compute01 | manual-management | | nuc-compute02 | manual-management | +------------------+--------+ 1 rows in set (0.00 sec)

After all this you can now safely continue with the common installation process, just remember when performing Overcloud deployment, check the node status with the ironic node-list command. Wait until the node status changes from deploying to deploy wait-callback and then manually power the nodes.