Some background: I am transitioning from using R + RStudio for all of my data analytic tasks to trying to use Python (pandas, mostly) + Eclipse with PyDev. Because of RStudio, I am accustomed to a workflow that involves writing some code, and testing sections I'm not sure about by running it chunk by chunk, making any fixes I need to do and then moving on. RStudio really facilitates (actually, probably trained me) to do this and since I don't have any formal CS training, this is how I think now.
I have two questions (one practical, the other philosophical):
1.) How can you get the interactive console in Eclipse + PyDev to recognize variables in your larger script? I know that ctrl + alt + enter will bring up the interactive console, but I can neither make it stay once I have moved back to tinkering within my script, and it also doesn't recognize anything that's been run before. So if I want to test the use of a function I have defined on a specific column of data, I have to copy/paste the function into the console (as well as any dependencies, like importing modules).
2.) What workflow do Python users do instead? I get the sense that this reliance upon manual guess and check testing isn't what people who actually program do, and I would like to transition into a better practice. But so far I am finding that rerunning the whole script from the beginning after I make changes is kind of a waste, when I'd rather just hone in on the sections I am actively working on.
After you do Ctrl+Alt+Enter, you should be able to:
Make Ctrl+Alt+Enter to execute the whole file on the console so that you can use its functions.
Put the cursor on any line and press F2 so that it'll execute that line on the console and will go to the next (so, you may write a block of code by pressing F2 to execute line after line).
Note that if you want, you may see the local variables from the shell if you checked 'Connect console to a Debug Session?' at the preferences > pydev > interactive console page.
Now, this is a reasonable approach for exploratory testing with slow data (where you usually load slow data and make changes with the data in memory), but for code which is fast, what I suggest is creating test cases and then running those tests (I recommend using pytest: http://pytest.org, but unittest works too)... and in that case, you also can have tests stored for posterity which you can rerun to make sure you didn't break anything as your code evolves.
Note that if you decide to use pytest, you have to go to the preferences > PyDev > PyUnit and set the test runner to PyTest. Note that PyDev makes it pretty straightforward to run just the test that you want (you can select the test method with Ctrl+Shift+Up/Down in the test file and then use Ctrl+F9 to run it... and if you want, you can just use Ctrl+F11 to rerun that later on or F11 to debug it with breakpoints, provided you configure the "Always launch the previously launched application" as described in http://www.pydev.org/manual_101_run.html).