Ansible Tasks reuse of HTTP Session from Python Requests module

Question

Ansible Tasks reuse of HTTP Session from Python Requests

Scenario:

• Complex login process to Web Service with multiple re-directs and form scrapes (via BeautifulSoup)
• Python script file using Requests module with a Session, to store tokens and cookies of the success for login for subsequent Web Service API calls

Objective:

• Ansible Task 1: Execute Python script for Web Service authentication, return HTTPS Session to Ansible
• Ansible Task 2..n: Execute Web Service APIs using built-in uri Ansible Module, using the HTTPS Session from the

Thoughts:

• Honestly I am a little confused how to structure this effectively in Ansible. It might be sensible to consider the entire Python script for the authentication as an Ansible Module?

---

EXAMPLE PYTHON AUTHENTICATE SCRIPT (simplified)

def main():
    import requests

    # Create Session for requests, store all cookies in this object
    https_session = requests.Session()

    if 'page_1' not in globals():
        page_1 = https_session.get(
            'https://login.test.com',
            allow_redirects=True
        )

    ...
    ...
    ...

    if 'redirect_8' not in globals():
        redirect_8 = https_session.post(
            'https://login.test.com/authenticate',
            allow_redirects=True,
        data = {
            "token": response_7['token']
            }
        )

    # Return the Cookie Jar
    return https_session.cookies


main()

EXAMPLE ANSIBLE CALL:

- name: test1
  script: login.py
  args:
        executable: python3
  environment:
        username: "{{ input_user }}"
        password: "{{ input_pwd }}"
  ignore_errors: yes
  register: login_test

- debug:
    msg: "{{ login_test }}.stdout"

EXAMPLE RETURN TO ANSIBLE:

"
{
    "results": [
        {
            "changed": True,
            "rc": 0,
            "stdout": "<RequestsCookieJar[<Cookie TOKEN=wxyz-wxyz for .home.test.com/>, <Cookie USER_ID=123456789 for .home.test.com/>\\n",
            "stdout_lines": [
                "<RequestsCookieJar[<Cookie TOKEN=wxyz-wxyz for .home.test.com/>, <Cookie USER_ID=123456789 for .home.test.com/>, "
            ],
            "stderr": "",
            "stderr_lines": [],
            "failed": False,
            "ansible_loop_var": "item",
        }
    ],
    "skipped": False,
    "changed": True,
    "msg": "All items completed",
}
.stdout"

Answer 1

This could be wrapped nicely in a lookup plugin:

lookup_plugins/session_cookie.py:

from __future__ import (absolute_import, division, print_function)
__metaclass__ = type

DOCUMENTATION = """
  lookup: session_cookie
  author: Author
  options:
    username:
      type: string
      required: True
    password:
      type: string
      required: True
"""

from ansible.plugins.lookup import LookupBase
from ansible.utils.display import Display
import requests

display = Display()

class LookupModule(LookupBase):
    def run(self, _terms, variables=None, **kwargs):
        self.set_options(var_options=variables, direct=kwargs)

        url_username=self.get_option('username')
        url_password=self.get_option('password')

        https_session = requests.Session()

        # do your custom sutff here
        https_session.get(
            'https://google.com',
            allow_redirects=True
        )

        # retrieve cookies from session an return them
        cookies = https_session.cookies.get_dict()

        return [cookies]

And then it can be used in playbook like this:

playbook.yml

- name: Get session cookie
  hosts: localhost
  connection: local
  gather_facts: false
  vars:
    password: secret
    user: admin
    cookie: "{{ lookup('session_cookie', username=user, password=password) }}"
  tasks:
    - debug:
         msg: "{{ cookie['1P_JAR'] }}"

Running it would return cookie value from the session:

ansible-playbook -i',' playbook.yaml
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [Get session cookie] *****************************************************************************************************

TASK [debug] ******************************************************************************************************************
ok: [localhost] => {
    "msg": "2021-09-13-10"
}

I've noticed that requests sometimes did not play nicely with ansible forking, see ansible issue, so I had to add this env export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES.