Unable to load python packages when creating a system file in linux

Question

I have created a simple script to send message to AWS SQS from a python file. I have imported boto3 package and tested it locally. Now when I am trying to create a service file and run it using it, it says unable to import boto3, even though the boto3 package is installed (I rechecked it using pip freeze | grep boto3)

Here is my code:

import boto3
import json
AWS_SQS_QUEUE_NAME = "XXXXXXXXX"


class SQSQueue(object):
    def __init__(self, queueName=None):
        self.resource = boto3.resource('sqs',region_name="ap-south-1")
        self.queue = self.resource.get_queue_by_name(QueueName=AWS_SQS_QUEUE_NAME)
        self.QueueName = queueName
    
    def send(self, Message={}):
        data = json.dumps(Message)
        response = self.queue.send_message(MessageBody=data)
        return response
    
if __name__ == '__main__':
    q = SQSQueue(queueName=AWS_SQS_QUEUE_NAME)
    message = {
        "user": "XXXX",
        "timestamp": "2020-03-04 18:06:09"
    }
    
    response = q.send(Message=message)

My Service file

[Unit]
Description=Dummy Service
After=multi-user.target

[Service]
Type=simple
User=root
Group=root
ExecStart=/usr/bin/python3 /home/ubuntu/code/sample.py

[Install]
WantedBy=multi-user.target

I also tried creating a bash file and calling the python file from there but it gives the same error

Error:

Sep 04 15:31:34 virtualhost systemd[1]: Started Access Service.
Sep 04 15:31:35 virtualhost python3[6877]: Traceback (most recent call last):
Sep 04 15:31:35 virtualhost python3[6877]:   File "/home/ubuntu/code/sample.py", line 1, in <module>
Sep 04 15:31:35 virtualhost python3[6877]:     import boto3
Sep 04 15:31:35 virtualhost python3[6877]: ModuleNotFoundError: No module named 'boto3'
Sep 04 15:31:35 virtualhost systemd[1]: sudoaccess.service: Main process exited, code=exited, status=1/FAILURE
Sep 04 15:31:35 virtualhost systemd[1]: sudoaccess.service: Failed with result 'exit-code'.

Answer 1

The issue is you have boto3 installed as a user package, and you're running the systemd service as root, this is why Python only has access to system level packages.

You have 2 options:

The easy way is installing boto3 system wide, by running python3 -m pip install boto3 as root.

Or a much cleaner way for a deployment would be create a virtualenv with only the required libraries:

python3 -m venv /home/ubuntu/myenv/
source /home/ubuntu/myenv/bin/activate
pip install boto3

And then use that virtualenv on the service definition:

[Service]
Type=simple
User=ubuntu
Group=ubuntu
ExecStart=/home/ubuntu/myenv/bin/python /home/ubuntu/code/sample.py