Search code examples
pythonpython-3.xsublimetext3sublimetextsublime-text-plugin

Global Python packages in Sublime Text plugin development

1. Summary

I don't find, how Sublime Text plugins developer can use Sublime Text find global Python packages, not Python packages of Sublime Text directory.

Sublime Text use own Python environment, not Python environment of machine. Developers needs sys.path for set not built-in Sublime Text Python packages.

Is any methods, that use global installed Python packages in Sublime Text plugins? For example, it would be nice, if someone tells me, how I can change my plugin — see 3.2 item of this question.

2. Disadvantages of using Sublime Text 3 environment

  1. Sublime Text 3 Build 3126 use Python 3.3, but at the time of writing this question release Python 3.6 stable. Python 3.6 have more features.
  2. Developers needs add and update third-party Python packages, even they installed for users. It spends a time of developers.
  3. For developers there may be problems with dependencies of packages, see 6.2 item of this question.

3. Example

1. Python code

For example, I wrote Python code — replace Поиск Кристиниты to [**Поиск Кристиниты**](https://github.com/Kristinita/Kristinita.github.io), where https://github.com/Kristinita/Kristinita.github.io — first link of DuckDuckGo query Поиск Кристиниты.

# -*- coding: utf-8 -*-
import re
import urllib

from bs4 import BeautifulSoup

from w3lib.url import safe_url_string


# ASCII link for solved encoding problems —
# https://stackoverflow.com/a/40654295/5951529
ascii_link = safe_url_string(
    u'http://duckduckgo.com/html/?q=' + 'Поиск Кристиниты',
    encoding="UTF-8")
print(ascii_link)
# SERP DuckDuckGo
serp = urllib.request.urlopen(ascii_link)
# Reading SERP
read_serp = serp.read()
# BeautifulSoup — https://stackoverflow.com/a/11923803/5951529
parsed = BeautifulSoup(read_serp, "lxml")
# Parsed first link
first_link = parsed.findAll(
    'div', {'class': re.compile('links_main*')})[0].a['href']
# Remove DuckDuckGo specific characters —
# https://stackoverflow.com/a/3942100/5951529
remove_duckduckgo_symbols = first_link.replace("/l/?kh=-1&uddg=", "")
# https://stackoverflow.com/a/32451970/5951529
final_link = (urllib.parse.unquote(remove_duckduckgo_symbols))
# Markdown link
markdown_link = '[' + 'Поиск Кристиниты' + ']' + \
    '(' + final_link + ')'

print(markdown_link)

If I run this file in terminal or SublimeREPL, I get in output:

[**Поиск Кристиниты**](https://github.com/Kristinita/Kristinita.github.io/)

2. Sublime Text plugin

Now, based on this code, I wrote Sublime Text plugin for replace example text to [**example text**](http://<first link for DuckDuckGo query “example link”>):

import re
import urllib

from bs4 import BeautifulSoup

from w3lib.url import safe_url_string

import sublime_plugin


class KristinitaLuckyLinkCommand(sublime_plugin.TextCommand):

    def run(self, edit):
        # Get selection text
        print('KristinitaLuckyLink called')
        select = self.view.sel()
        selection_region = select[0]
        selection_text = self.view.substr(selection_region)
        print(selection_text)

        # ASCII link for solved encoding problems —
        # https://stackoverflow.com/a/40654295/5951529
        ascii_link = safe_url_string(
            u'http://duckduckgo.com/html/?q=' + (selection_text),
            encoding="UTF-8")
        print(ascii_link)
        # SERP DuckDuckGo
        serp = urllib.request.urlopen(ascii_link)
        # Reading SERP
        read_serp = serp.read()
        # BeautifulSoup — https://stackoverflow.com/a/11923803/5951529
        parsed = BeautifulSoup(read_serp, "lxml")
        # Parsed first link
        first_link = parsed.findAll(
            'div', {'class': re.compile('links_main*')})[0].a['href']
        # Remove DuckDuckGo specific characters —
        # https://stackoverflow.com/a/3942100/5951529
        remove_duckduckgo_symbols = first_link.replace("/l/?kh=-1&uddg=", "")
        # Final link — https://stackoverflow.com/a/32451970/5951529
        final_link = (urllib.parse.unquote(remove_duckduckgo_symbols))
        markdown_link = '[' + selection_text + ']' + \
            '(' + final_link + ')'
        print(markdown_link)

        # Replace selected text to Markdown link
        self.view.replace(
            edit, selection_region, markdown_link)

4. Expected behavior

If user have installed Python and install packages

  • pip install beautifulsoup4
  • pip install lxml
  • pip install w3lib

I want, that my plugin from 2.2 item successful work for user.

5. Actual behavior

If I save my plugin, I get stack trace:

Traceback (most recent call last):
  File "D:\Sublime Text Build 3126 x64 For Debug\sublime_plugin.py", line 109, in reload_plugin
    m = importlib.import_module(modulename)
  File "./python3.3/importlib/__init__.py", line 90, in import_module
  File "<frozen importlib._bootstrap>", line 1584, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 584, in _check_name_wrapper
  File "<frozen importlib._bootstrap>", line 1022, in load_module
  File "<frozen importlib._bootstrap>", line 1003, in load_module
  File "<frozen importlib._bootstrap>", line 560, in module_for_loader_wrapper
  File "<frozen importlib._bootstrap>", line 868, in _load_module
  File "<frozen importlib._bootstrap>", line 313, in _call_with_frames_removed
  File "D:\Sublime Text Build 3126 x64 For Debug\Data\Packages\Grace Splitter\kristi.py", line 4, in <module>
    from bs4 import BeautifulSoup
ImportError: No module named 'bs4'

6. Not helped

1. Using global Python environment of computer

I don't find, how I can do it. Examples of questions, that I can find:

2. Using Sublime Text environment

I install

I copy my w3lib directory from C:\Python36\Lib\site-packages to Data\Packages directory of Sublime Text.

I run in Sublime Text 3 console:

>>> window.run_command("kristinita_lucky_link")

I get stack trace:

Traceback (most recent call last):
  File "D:\Sublime Text 3 x64\sublime_plugin.py", line 818, in run_
    return self.run(edit)
  File "D:\Sublime Text 3 x64\Data\Packages\KristinitaLuckyLink\KristinitaLuckyLink.py", line 32, in run
    parsed = BeautifulSoup(read_serp, "lxml")
  File "D:\Sublime Text 3 x64\Data\Packages\bs4\__init__.py", line 165, in __init__
    % ",".join(features))
bs4.FeatureNotFound: Couldn't find a tree builder with the features you requested: lxml. Do you need to install a parser library?

I don't find, how I can set lxml.

3. Using variables in 2 files

For example I have KristinitaLuckyLink.py and KrisDuckDuckGo.py files in the same directory.

My KristinitaLuckyLink.py file:

import re
import requests
import sublime_plugin
import subprocess
import sys
sys.path.append(
    'D:\Sublime Text 3 x64\Data\Packages\KristinitaLuckyLink\KrisDuckDuckGo.py')

from KrisDuckDuckGo import final_link
from bs4 import BeautifulSoup


class KristinitaLuckyLinkCommand(sublime_plugin.TextCommand):

    def run(self, edit):
        # Get selection text
        print('KristinitaLuckyLink called')
        select = self.view.sel()
        selection_region = select[0]
        selection_text = self.view.substr(selection_region)
        print(selection_text)

        # Get terminal output — https://stackoverflow.com/a/4760517/5951529
        # Paths is correct
        result = subprocess.run(["C:\Python36\python.exe", "D:\Sublime Text 3 x64\Data\Packages\KristinitaLuckyLink\krisduckduckgo.py"],
                                stdout=subprocess.PIPE)
        final_link = result.stdout.decode('utf-8')
        print(final_link)

        # Markdown link
        markdown_link = '[' + selection_text + ']' + \
            '(' + final_link + ')'
        print(markdown_link)

        # Replace selected text to Markdown link
        self.view.replace(
            edit, selection_region, markdown_link)

My KrisDuckDuckGo.py file:

import urllib

import sys
sys.path.append(
    'D:\Sublime Text 3 x64\Data\Packages\KristinitaLuckyLink\KristinitaLuckyLink.py')

from w3lib.url import safe_url_string

from KristinitaLuckyLink import selection_text

from bs4 import BeautifulSoup


# ASCII link for solved encoding problems —
# https://stackoverflow.com/a/40654295/5951529
ascii_link = safe_url_string(
    u'http://duckduckgo.com/html/?q=' + (selection_text),
    encoding="UTF-8")
print(ascii_link)
# SERP DuckDuckGo
serp = urllib.request.urlopen(ascii_link)
# Reading SERP
read_serp = serp.read()
# BeautifulSoup — https://stackoverflow.com/a/11923803/5951529
parsed = BeautifulSoup(read_serp, "lxml")
# Parsed first link
first_link = parsed.findAll(
    'div', {'class': re.compile('links_main*')})[0].a['href']
# Remove DuckDuckGo specific characters —
# https://stackoverflow.com/a/3942100/5951529
remove_duckduckgo_symbols = first_link.replace("/l/?kh=-1&uddg=", "")
# Final link — https://stackoverflow.com/a/32451970/5951529
final_link = (urllib.parse.unquote(remove_duckduckgo_symbols))
print(final_link)

I select any text → I print in Sublime Text console:

window.run_command("kristinita_lucky_link")

I don't get output in Sublime Text console.

7. Environment

Operating system and version:
Windows 10 Enterprise LTSB 64-bit EN
Sublime Text:
Build 3126
Python:
3.6.0

Solution

  • Yes, plugins can work with Python modules, that user install globally. You are not obligated introduce modules to your Sublime Text plugin.

    Problems

    1. Python 3.3 compatibility

    For begin of 2018:

        All global modules, that you use in your plugins, must be compatible with Python 3.3.

    1.1. Argumentation

    1.1.1. Sublime Text sys.path order

    For example, I add to Default.sublime-package archive file 0000.py. Modules from Default.sublime-package are loaded first by Sublime Text start.

    0000.py content:

    import sys

sys.path.append('C:\\Python36')
sys.path.append('C:\\Python36\\python36.zip')
sys.path.append('C:\\Python36\\DLLs')
sys.path.append('C:\\Python36\\lib')
sys.path.append('C:\\Python36\\lib\\site-packages')

    Where paths — my global sys.path.

    >>> import sys; sys.path
['', 'C:\\Python36', 'C:\\Python36\\python36.zip', 'C:\\Python36\\DLLs', 'C:\\Python36\\lib', 'C:\\Python36\\lib\\site-packages']

    I restart Sublime Text → I see in console:

    DPI scale: 1
startup, version: 3143 windows x64 channel: stable
executable: /D/Sublime Text Build 3143 x64 For Debug/sublime_text.exe
working dir: /D/Kristinita
packages path: /D/Sublime Text Build 3143 x64 For Debug/Data/Packages
state path: /D/Sublime Text Build 3143 x64 For Debug/Data/Local
zip path: /D/Sublime Text Build 3143 x64 For Debug/Packages
zip path: /D/Sublime Text Build 3143 x64 For Debug/Data/Installed Packages
ignored_packages: ["Anaconda", "Vintage"]
pre session restore time: 0.458819
startup time: 0.493818
first paint time: 0.506818
reloading plugin Default.0000
reloading plugin Default.auto_indent_tag
reloading plugin Default.block
# And so on

    >>> import sys; sys.path
['D:\\Sublime Text Build 3143 x64 For Debug', 'D:\\Sublime Text Build 3143 x64 For Debug\\python3.3.zip', 'D:\\Sublime Text Build 3143 x64 For Debug\\Data\\Lib\\python3.3', 'D:\\Sublime Text Build 3143 x64 For Debug\\Data\\Packages', 'C:\\Python36', 'C:\\Python36\\python36.zip', 'C:\\Python36\\DLLs', 'C:\\Python36\\lib', 'C:\\Python36\\lib\\site-packages']

    Paths from the internal environment are in front, that paths in global environment. I don't know, how I can change it.

    1.1.2. Python 3.6 module

    For example, I want to use in my plugin Python Google Search API PyPI module. It compatible with Python 3.6, but not compatible with Python 3.3.

    I inject file 0000.py as in subsection above → I create plugin SashaGSearch.py, it content:

    import sublime_plugin

from gsearch.googlesearch import search


class GoogleSearchCommand(sublime_plugin.TextCommand):

    def run(self, edit):

        results = search('kristinitaluckylife', num_results=1)
        r = results[0][1]
        print(r)

    I restart Sublime Text → I get same traceback, as if 0000.py not be implemented.

    1.2. Situation on begin of 2018

    See Sublime Text forum discussions:

    I hope, that in internal Sublime Text environment Python 3.3 will be replaced to 3.6 or next 3.7 version in the nearest future.

    2. Environment variable

    If you know, that possible don't add new environment variable, please, answer to this question.

    2.1. Plugin code

    You need to create on your PC environment variable, where value — path to site-packages folder. For example, I named it PYTHONPACKAGES:

    You need add to your code lines as in OdatNurd answer:

    # PYTHONPACKAGES path:
# https://stackoverflow.com/a/4907053/5951529
# Disable duplicate paths:
# https://stackoverflow.com/a/42656754/5951529
site_packages = (os.environ['PYTHONPACKAGES'])
if site_packages not in sys.path:
    sys.path.append(site_packages)

    2.2. Activation instructions

    Users of your plugin must also add environment variable PYTHONPACKAGES in operating system. In description of your package, you need add instructions, how do it.

    3. Restart

    For developing process:

        You may need to restart Sublime Text, if you change files in site-packages folder or subfolders.

    3.1. Example

    You install examplesashamodule from pip → you start Sublime Text → you import examplesashamodule to your plugin → you delete examplesashamodule from site-packages folder → plugin will work as examplesashamodule is still in the site-packages folder. AutomaticPackageReloader package doesn't help in this case.

    You restart Sublime Text → you get traceback in Sublime Text console, that ImportError: No module named 'examplesashamodule'.

    Apparently, Sublime Text cache data from external modules in a session.