LookupError: 'base64' is not a text encoding when upgrading code from Python2 to Python3

Question

I have a piece of Python2 code that i would like to upgrade to Python3. It is a Cinder storage driver that communicates with and API.

Most of the code i could 'convert' myself or with the help of 2to3. I'm no programmer, but am willing to learn. I am stuck at the following piece of code:

def _create_request(self, request_d, param_list):
    """Creates urllib.Request object."""
    if not self._username or not self._password:
        raise ValueError("Invalid username/password combination")
    auth = ('%s:%s' % (self._username,
                       self._password)).encode('base64')[:-1]
    headers = {'Content-Type': 'application/json',
               'Authorization': 'Basic %s' % (auth,)}
    url = self.get_url() + request_d
    LOG.debug('url : %s', url)
    LOG.debug('param list : %s', param_list)
    return urllib.request.Request(url, param_list, headers)

specifically this part:

    auth = ('%s:%s' % (self._username,
                       self._password)).encode('base64')[:-1]

which gives me the following error:

LookupError: 'base64' is not a text encoding; use codecs.encode() to handle arbitrary codecs

I have tried a few options, like base64.b64encode, but lack the knowledge to convert it properly. Please be aware that it is a string to be base64 encoded.

this is my best bet:

import base64    
auth = base64.b64encode(('%s:%s' % (self._username, self._password)).encode('ascii'))

Is this correct? If not, what would be the good approach?

Update #1: I tried some tests outside of this code and i'm getting closer:

python2
>>> _username = 'test'
>>> _password = 'test'
>>> auth = ('%s:%s' % (_username, _password)).encode('base64')[:-1]
>>> print (auth)
dGVzdDp0ZXN0

python3
>>> import base64
>>> _username = 'test'
>>> _password = 'test'  
>>> auth = base64.b64encode(('%s:%s' % (_username, 
_password)).encode('ascii'))
>>> print (auth)
b'dGVzdDp0ZXN0'

As you can see the desired output, as per Python2 output, does not yet match the output of the Python3 code.

Update #2: I found a working solutions with the following code:

import base64
auth = base64.urlsafe_b64encode(('%s:%s' % (self._username, self._password)).encode('utf-8'))
authstr = str(auth, "utf-8")

And i'm starting to hurt myself. The output is a str and should be bytes...

Answer 1

In python2, bytes and str were equivalent, but no longer with python3.

From '%s:%s' % (self._username, self._password) you get a str (e.g. 'test:test'). The base64.b64encode method (and related methods) requires bytes, and this is why you must encode first.

base64.b64encode will return a bytes value (e.g. b'dGVzdDp0ZXN0'). If you want to get back to a str you must then decode, with either str(xyz, 'utf-8') or xyz.decode('utf-8').

Also, utf-8 is the default for str encode and bytes decode.

import base64
auth = base64.urlsafe_b64encode(('%s:%s' % (self._username,
                                            self._password)).encode()).decode()