Search code examples
pythoneclipsedebuggingpydevbreakpoints

Ecplipse, PyDev, Django: Breakpoints in settings.py work, but in views do not

I've embarked on a rather ambitious project in my dev environment because I like keeping up to date I guess. So I've been working on a Django project with Eclipse and PyDev for years and it works beautifully.

I've been working with Python 3.7 an Django 2.1 and Eclipse Photon for some while.

I've tried a kind of complete new start of sorts as follows moving to latest version of everything:

  1. Installed latest Eclipse (2019-12 Eclipse IDE for Web and JavaScript Developers (includes Incubating components)
  2. Add PyDev to it (Help > Install New Software, from http://www.pydev.org/updates)
  3. Installed Python 3.8.
  4. Installed Django 3.0.1.
  5. Installed all the packages my project depends on into my local 3.8 site-packages
  6. In a new workspace I imported my old project (File > Import > General > Existing Projects into Workspace, checked Copy projects into workspace
  7. Created a debug configuration with the Python 3.8 interpreter and (because for some reason the default sys.path put my local site-packages after python 3 dist-packages) I added site-packages to my external libraries and forced it to the top of my sys.path.
  8. Started a Debug run ...

This works fine. It starts, I can work with my website pointing at http://127.0.0.1:8000/ and it's mostly functional, in fact can't complain, almost expected more dramas in upgrading. Not perfect had a few tweaks to make to get it Django 3.0.1 compatible but surprisingly few.

I am stuck on page which is crashing though and so I set a breakpoint only to discover it's not working ... Hmmmm. I have been here before with PyDev and so have many others. So I have performed the basic diags already and will share what I know. But I would be most grateful if someone (not least Fabio) had a further gem of wisdom to lend to get them working.

Installed on Eclipse are:

  PyDev for Eclipse 7.4.0.201910251334  org.python.pydev.feature.feature.group  Fabio Zadrozny
  PyDev for Eclipse Developer Resources 7.4.0.201910251334  org.python.pydev.feature.source.feature.group   Fabio Zadrozny
  Pydev Mylyn Integration   0.6.0   org.python.pydev.mylyn.feature.feature.group    Fabio Zadrozny

Key Observations:

  1. If I set a breakpoint in settings.py it breaks fine.
  2. If I set a breakpoint in a view it does not break.
  3. I know I'm in the right file and place because if I add a console print line at the place I want to break, it is printed to the console. So I'm not looking at the wrong source file in any way shape or form.

Fabio left some really nice (dated now) tips here: pydev breakpoints not working and I can say in response to those:

  1. If I put this at top of views.py where Iw ant my breakpoint: 
import sys
print('DEBUG: current trace function', sys.gettrace())

I see this output:

DEBUG: current trace function None

which seems to be the problem. That something is turning off debugging.

Following Fabio's first tip and accounting for changes in PyDev structure since that tip, I found pydevd_constants.py in plugins/org.python.pydev.core_7.4.0.201910251334/pysrc/_pydevd_bundle and in it could set:

DEBUG_TRACE_LEVEL = 3 
DEBUG_TRACE_BREAKPOINTS = 3

As Fabio suggested. This reveals little alas. Essentially I see lots of cool tracing when start the debug run until the server is running. Then I see the standard output saying the sever is now running:

System check identified no issues (0 silenced).
December 26, 2019 - 12:29:54
Django version 3.0.1, using settings 'MyProject.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

It stops on the breakpoint in settings.py fine along the way and I continue with F8.

Now in my browser I load the view I'm debugging. And there and zero PyDev traces on the console alas, in fact I get only my little printed line I put int he view's get_queryset() method to ensure my code (with the breakpoint) is running.

So that didn't help any. I can in my views.py file then look for a needle int he haystack with Fabio's suggestion:

import sys
print('DEBUG: current trace function', sys.gettrace())
def trace_func(frame, event, arg):
    with open('pydev-trace.txt', 'a') as f:
        print('Context: ', frame.f_code.co_name, '\tFile:', frame.f_code.co_filename, '\tLine:', frame.f_lineno, '\tEvent:', event, file=f)
    return trace_func

sys.settrace(trace_func)
print('DEBUG: current trace function', sys.gettrace())

and on the console I now see:

DEBUG: current trace function None
DEBUG: current trace function <function trace_func at 0x7ffaa6f18b80>
System check identified no issues (0 silenced).
December 26, 2019 - 12:42:48
Django version 3.0.1, using settings 'MyProject.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Which sort of looks good.

But the output trace seems never to stop. It grows and grows and grows and seems stuck in a cycle of autoreloads in an endlessly looping cycle of:

Context:  snapshot_files    File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 370   Event: call
Context:  snapshot_files    File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 361   Event: line
Context:  watched_files     File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 260   Event: call
Context:  watched_files     File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 260   Event: return
Context:  snapshot_files    File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 362   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 758   Event: call
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 759   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 760   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 760   Event: return
Context:  snapshot_files    File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 364   Event: line
Context:  snapshot_files    File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 365   Event: line
Context:  stat  File: /usr/lib/python3.8/pathlib.py     Line: 1186  Event: call
Context:  stat  File: /usr/lib/python3.8/pathlib.py     Line: 1191  Event: line
Context:  __fspath__    File: /usr/lib/python3.8/pathlib.py     Line: 721   Event: call
Context:  __fspath__    File: /usr/lib/python3.8/pathlib.py     Line: 722   Event: line
Context:  __str__   File: /usr/lib/python3.8/pathlib.py     Line: 711   Event: call
Context:  __str__   File: /usr/lib/python3.8/pathlib.py     Line: 714   Event: line
Context:  __str__   File: /usr/lib/python3.8/pathlib.py     Line: 715   Event: line
Context:  __str__   File: /usr/lib/python3.8/pathlib.py     Line: 715   Event: return
Context:  __fspath__    File: /usr/lib/python3.8/pathlib.py     Line: 722   Event: return
Context:  stat  File: /usr/lib/python3.8/pathlib.py     Line: 1191  Event: return
Context:  snapshot_files    File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 369   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 758   Event: call
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 759   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 760   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 760   Event: return
Context:  snapshot_files    File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 370   Event: line
Context:  snapshot_files    File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 370   Event: return
Context:  tick  File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 346   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 758   Event: call
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 759   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 760   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 760   Event: return
Context:  __eq__    File: /usr/lib/python3.8/pathlib.py     Line: 753   Event: call
Context:  __eq__    File: /usr/lib/python3.8/pathlib.py     Line: 754   Event: line
Context:  __eq__    File: /usr/lib/python3.8/pathlib.py     Line: 756   Event: line
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 744   Event: call
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 747   Event: line
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 748   Event: line
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 748   Event: return
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 744   Event: call
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 747   Event: line
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 748   Event: line
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 748   Event: return
Context:  __eq__    File: /usr/lib/python3.8/pathlib.py     Line: 756   Event: return
Context:  tick  File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 347   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 758   Event: call
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 759   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 760   Event: line
Context:  __hash__  File: /usr/lib/python3.8/pathlib.py     Line: 760   Event: return
Context:  __eq__    File: /usr/lib/python3.8/pathlib.py     Line: 753   Event: call
Context:  __eq__    File: /usr/lib/python3.8/pathlib.py     Line: 754   Event: line
Context:  __eq__    File: /usr/lib/python3.8/pathlib.py     Line: 756   Event: line
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 744   Event: call
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 747   Event: line
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 748   Event: line
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 748   Event: return
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 744   Event: call
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 747   Event: line
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 748   Event: line
Context:  _cparts   File: /usr/lib/python3.8/pathlib.py     Line: 748   Event: return
Context:  __eq__    File: /usr/lib/python3.8/pathlib.py     Line: 756   Event: return
Context:  tick  File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 348   Event: line
Context:  tick  File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 351   Event: line
Context:  tick  File: /home/me/.local/lib/python3.8/site-packages/django/utils/autoreload.py    Line: 345   Event: line

and that is the best clue we have to date. I suspect something here is in a very tight unending loop. Still the webpage loads so it's not draining resources at a breakneck speed and it runs for ages and doesn't overflow any stacks or anything but clearly seems broken somehow.

But it's getting very hard to pin something down here.

Finally in that response Fabio suggested that a brute force fix might be:

import pydevd;
pydevd.settrace()

Alas could not import pydevd, but if I add to my External libraries:

...plugins/org.python.pydev.core_7.4.0.201910251334/pysrc

I can, and then run it but it produces an endless stream of text on my console and my website won't respond while that's happening. So it's brute force in the sense of brining by debug server down ;-). This may relate to endless cycle above.

Other checks:

I checked Fabios list of needs:

http://www.pydev.org/manual_adv_django.html

and on those I'm good.

  1. The project is marked as a Django project.
  2. DJANGO_MANAGE_LOCATION is set.
  3. DJANGO_SETTINGS_MODULE is set.

I can see the latter two on Project Properties > PyDev PYTONPATH > String Substitution Variables. And I cna infer the first from the presence of Django menus on the projects context Menu and absense of the option to Set as Django Project.

At this point I'm hoping for some further pointers. I have invest a good deal of time into diagnosis here and trying to uncover a cause but my efficiency at this point will skyrocket with some injected expertise and advice. Fabio?

I am asking here, in no small part because many otehrs have, and it is a superbly useful well findable resource for anyone facing a similar problem. And there's no reason to believe that unlikely given, the circumstances. So I'd like this solved, and a solution posted basically for my sake and others'.

Solution

  • PyDev release 7.5.0 which appeared on Jan 10 2020, fixed this issue. Breakpoints in my views function properly once more. The release notes I have include:

    • Fixed support for Python 3.8 (which wasn't properly added to 7.4.0).
    • Debugger improvements (updated to pydevd 1.9.0).
      • Improved stepping into coroutines
      • Attach to process no longer needs to match the bitness of the target program with the interpreter.
      • File with a relative path is searched in sys.path folders (i.e.: so, cython builds can find the source).

    Leading me to suspect that the whole issue related to Python 3.8 (which would be far from surprising given the very recent release of Python 3.8) though it's possible tweaks in pydevd 1.9.0 played a role. Either way a PyDev update to 7.5.0 fixes the issue!