Notice: Community Accounts Have Changed. What You Need To Know >>

Topic - Remote debugging Python in modo 701+

Page 1 of 2
   1  2  Next  Last 
Here's some remote debugging information for Python in modo 701. This info will also be in the SDK wiki once that is updated but I thought it was important/useful enough that it was worth posting ASAP.

In 701 there are now essentially two distinct 'paths', or 'modes' of execution for python scripts. The old, pre-701 style scripts that are executed using the '@' syntax or the 'script.run' command which run, as they have always done, through the command service and are transient or 'fire & forget' scripts and, new to 701, we have python 'plugins' which register themselves into the system at start up and stay resident throughout the session, just like any regular plugin written with the SDK. The code for these new 'plugins' only executes when the functionality it defines is accessed or triggered by a user action such as a file save, for example. The differences in the way that these two routes operate means that there are some slight differences in the way that each one needs to be set up for debugging and/or the debug process itself. For the purposes of the following discussion the term 'script' or 'python script' will refer to the old style scripts executed using the '@' syntax or 'script.run' and the the term 'plugin', or 'python plugin', will refer to the new style, persistent python scripts that register themselves as servers at start up.
Not all remote debuggers appear to work well, if at all with modo. Here's a short, not exhaustive list of tested graphical debuggers/IDEs:

Winpdb - working
ActiveState Komodo IDE - working
Wingware Wing IDE Personal - working
Wingware Wing IDE Proffessional - working

Eric4 - not working
JetBrains PyCharm - not working


WinPDB

Despite it's name WinPDB is a cross platform graphical and console debugger. It uses the wxPython runtime to generate it's UI so you will need that and a system (platform if you're on OSX) distribution of Python. It's available from the Winpdb website where you'll also find a link to the wxPython runtime files. Follow the instructions on the Winpdb site for installation. For debugging in modo it doesn't actually matter whether you install via setup.py or run as a standalone script, it works the same either way.

Setting up for use in modo
1) Copy the file 'rpdb2.py' from Winpdb'd installation folder to a location on modo's python path, ie to any valid imported folder, user:Scripts would be an obvious choice
2) Add an import directive to the top of the script to be debugged, importing 'rpdb2'
3) Add a line below the import to start the debugging engine:

rpdb2.start_embedded_debugger(password)

where 'password' is a plain text string of your choice. When it comes time to run the script and attach the debugger you will need to enter the same password into Winpdb's UI. More thorough coverage for this topic is available on the Winpdb website but one word of caution, do not use the interactive password command in your modo script, it will lock up modo requiring a force-quit.

Debugging a Python Script
Successfully debugging a python script relies quite heavily on the way the script is set up. For this demonstration I'm going to be using the following:


#!/usr/bin/env python
 
import lx
import traceback
import rpdb2
 
def start_debugger():
    rpdb2.start_embedded_debugger('mododebug')
 
def main():
    numitems = lx.eval('query sceneservice item.N ?')
    for x in range(numitems):
        id = lx.eval('query sceneservice item.id ? %s' % x)
        name = lx.eval('query sceneservice item.name ? %s' % id)
        type = lx.eval('query sceneservice item.type ? %s' % id)
        lx.out((id, name, type))
 
 
if __name__ == '__main__':
    try:
        main()
    except:
        lx.out(traceback.format_exc())


cont ....

Message edited by GwynneR on 3/26/2013 - 4:22 AM

... cont

The items to note here are that the Winpdb module (rpdb2) is imported at the top, the command to start the debugger service has been enclosed in a function call, 'start_debugger()', the main body of the script is also enclosed in a function call, 'main()' and we have the common python idiom to conditionally execute the main function depending on whether the script was imported or run directly. Why structure the script like this? Well when modo executes a script through the command system it reads the file from disk and passes it as one large string buffer to the python interpreter. Unfortunately, that means that Winpdb is unable to access the raw code and single stepping through the script is impossible. Wrapping the script in this way and importing it as a module allows Winpdb to query the physical location of the file on disk. Plus, as will become apparent, it affords some flexibility during the debugging process, instead of running the script directly, using the '@' syntax or 'script.run', we can take advantage of the new persistent and interactive interpreter in 701.

For a complete walk-through of remote debugging a python script with Winpdb see the following video.

Winpdb_script.mov


Debugging a Python plugin

For this demonstration I'll be the STL saver plugin which can be found in '/extra/scripts/lxserv' in modo's application folder.

Just as with the script debugging example we need to import the debug module (rpdb2) and wrap the command to start the debug server in a function call. This time, though, the reason is slightly different. If you study the example plugin file linked you should notice that there is no real 'body', it's just a collection of classes with a couple of calls to a function called lx.bless(). The 'bless()' function is responsible for registering the classes as servers within modo. Generally you're probably not going to want to have the debugger break in when the classes are being registered, moving the 'rpdb2.start_embedded_debugger()' command into a function body stops it being called when the plugin is first imported. If you do want to have the debugger break in during the 'blessing' stage then it's a simple matter to add a call to 'start_debugger()' just after the function definition.


#!/usr/bin/env python
 
import lx
import traceback
import rpdb2
 
def start_debugger():
    rpdb2.start_embedded_debugger('mododebug')


For a walk-through of remote debugging a python plugin with Winpdb see the following video:

Winpdb_plugin.mov


cont ...
... cont


Wingware Wing IDE (professional & Personal)


Remote dubugging of both python scripts and python plugins has been confirmed to work with Wing IDE Professional and Wing IDE personal.

Setting up for use in modo

Setting up is fairly straight forward, first the Wing IDE debug client module needs copying to a location on modo's Python path, ie to any valid imported folder such as 'user:Scripts'. The client module file (wingdbstub.py) can be found in the root of the Wing IDE application folder. You'll also need to copy the 'wingdebugpw' file from your Wing IDE user settings folder to the same location as 'wingdbstub.py'. You should be able to find your Wing IDE user settings folder in 'C:\Users\username\AppData\Roaming' on Windows, it will be called either 'Wing IDE 4' or 'Wing Personal 4' depending on which version you have installed.

Open 'wingdbstub.py' in a text editor and change 'kEmbedded = 0' to 'kEmbedded = 1'

Then, at the top of the script or plugin to be debugged, below any imports, add the following lines:


def debugger():
    import wingdbstub


That's all that is really required. For further information see section '3.13. Advanced Debugging Topics' in the Wing IDE help file.


Debugging a Python script

For a walk-through of remote debugging a python script with Wing IDE professional and Personal versions see the following video:

ScriptDebuggingWithWingIDEPro.mov


Debugging a Python plugin

For a walk-through of remote debugging a python plugin with Wing IDE professional and Personal versions see the following video:

DebugPluginWingIDE.mov


Debugging Hints & Tips

When debugging scripts you can make edits/updates to the script without exiting the debugger. Use the python 'reload(modulename)' function from modo's interactive python interpreter to update the script in memory and carry on debugging. Note that this is only true for scripts, not for plugins.

Piggybacking off of GwynneR's post!


PyCharm
PyCharm is a Python IDE by JetBrains, and is available from their site here: JetBrain's PyCharm
This walkthrough pertains to Python Scripts, I haven't tackled 701's Python Plugin authoring just yet.

Video Using PyCharm's Debugger with Modo 701

Setting up Pycharm for Modo
After installing PyCharm, you will need to create a Remote Debugger profile for Modo. Follow these steps to do that.
1) On PyCharm's top bar, select Run and choose Edit Configurations
2) Press the green + in the top left hand corner and choose Python Remote Debug
3) In the following window, name the configuration (Modo 701 for example) and make sure these options are set.
· Enabled: Single instance only
· Enabled: Redirect output to console (This will print Modo's Python output inside PyCharm's output window)
· Disabled: Suspend after connect (If this option is enabled, PyCharm will interrupt modo as soon as it makes the initial connection to PyCharm's debugger, not terribly useful in our situation. )


Copying PyCharm's Debugger Asset in to Modo's Scripts Directory
After you install PyCharm, its Debugger python assets are stored inside its installation directory. To make things easier, lets copy these in to Modo's script directory.

Browse to this location: C:\Program Files (x86)\JetBrains\PyCharm 2.7.1\helpers\
And copy this folder: pydev

In Modo, on the top menu bar choose System->Open User Scripts Folder. Paste the pydev folder in to this directory.
In my case, this directory is: C:\Users\nstevenson\AppData\Roaming\Luxology\Scripts


Connecting Modo to PyCharm's Listener
After you have configured PyCharm with a Modo debug profile and copied PyCharm's debugger modfules in to Modo's scripts location, you can now connect Modo and PyCharm together.
To start the debug listener, on PyCharm's top menu bar, choose Run and then select Debug Modo 701. Alternatively, you can click the green bug icon on the middle of the main menu bar.
The bottom section of PyCharm will expand and you should see that PyCharm is now listening for a connection.


With PyCharm Listening...
From inside Modo, open up a Command History panel and using the grey gear icon in the top right hand corner, switch its input to Python.
After that, enter these two lines in to the Python entry field:


from pydev import pydevd
pydevd.settrace('localhost', port=7720, stdoutToServer=True, stderrToServer=True, suspend=False)


If all goes well, PyCharm's listener should report that it is now connected to a pydev debugger. At this stage, you should be able to import python modules and use break points to debug your scripts.


cont ...

Message edited by Nicholas S. on 3/28/2013 - 8:50 PM

... cont


Your Own Scripts
If you have successfully connected Modo to PyCharm's debugger, you should now be able to set break points and debug your scripts without issue!


A Word of Caution...
There is currently a bug when using PyCharm's Debug Probe.
If you attempt to send a command to Modo using the Debug Probe, while Modo is halted with a break point, it will cause Modo to disconnect from PyCharm, requiring you to restart both applications before you can begin debugging again.

I have submitted this issue to JetBrain's bug tracker, so with luck this will eventually get remedied.



I hope this helps!

Message edited by Nicholas S. on 3/28/2013 - 8:22 AM

this is really awesome of you to write this step by step here !!!

I am really excited to see what will come out of this new 701 integration !

plugins galore !!!



PyCharm's video has been updated.

Hey OOZZEE! This was actually my favorite new feature of Modo, and I can't wait to jump in to Modo scripting.

And while I know Code is Code, and we could just be using text editors, having a nice IDE with code completion and a solid debugger workflow is so convenient. I hope my part helps helps :)

Message edited by Nicholas S. on 3/28/2013 - 9:16 PM

just watching the second video! really good.

you should make more tuts Nicholas. your voice is captivating !

i am really happy this is implemented too in modo !

cheers

Message edited by Keith. on 3/29/2013 - 7:04 AM

Thank you so much for these precious info!
Gwinner if you can, may you provide instructions on how to setup komodo too?
Can't wait to see the SDK wiki updated!
hey Mario, I just got Komodo's debugging working with 701.
I went the route of using Komodo's own remote debugger rather than WinPDB, the process is very similar to PyCharm and it's working great.

I'll do a quick video this evening!

and thank you OOZZEE! I really hope these help, my wife agrees with you regarding my voice, but I'm not so sure... ;)
Quote from Nicholas Stevenson :
hey Mario, I just got Komodo's debugging working with 701.
I went the route of using Komodo's own remote debugger rather than WinPDB, the process is very similar to PyCharm and it's working great.

I'll do a quick video this evening!

and thank you OOZZEE! I really hope these help, my wife agrees with you regarding my voice, but I'm not so sure... ;)

Thank you Nicholas!
The fact is that I never used a debugger before (I know, so lame... ), that's why my code is always full of trace and print commands :D
Nicholas, you should listen to your wife ! lol

seriously, there are so many requests for python tutorials around here so I think it would be more than welcomed to have someone do some.

cheers
The problem I find with Komodo's remote debugging (compared to, say, WingIDE) is that it's very clunky. You can't set breakpoints inside Komodo (or on the fly), you have to write them as 'break()' commands into your source script which, for one, means that to add or change a breakpoint when debugging a plugin you have to completely re-start modo, or if debugging a script, re-import it, because it constitutes a change to the underlying source code. That's pretty lame.

Message edited by GwynneR on 3/29/2013 - 12:07 PM

I agree GwynneR, that limitation does stink :(

for scripts it isn't so bad, but I didn't even think about that with plugins, that would be maddening. I really don't understand why Komodo's remote debugger doesn't handle break points, but I suppose Python is only a fraction of Komodo's use.

I switched from Wing because I found the font rendering (no anti aliasing) to be really harsh to look at, and the panel layout system was just really annoying to deal with. However, Wing 5 is being completely re-written to utilize QT, which has me very excited. Wing has always been an amazing Python editor, so I'm really looking forward to the next version. Which was targeted for October-ish of last year... =\
Quote from Nicholas Stevenson :However, Wing 5 is being completely re-written to utilize QT, which has me very excited. Wing has always been an amazing Python editor, so I'm really looking forward to the next version.
Oh really? That's interesting. There are quite a few things I don't like about Wing so I still write the raw code in Komodo and just use Wing for debugging
Indeed! I know that the font anti-aliasing and such is a limitation of the GTK UI kit its built on top of currently, so I really hope that with a new foundation built on QT, it will be a much sleeker looking and functioning editor.

My source on the QT rewrite was a wingware mailing thread response from one of the developers. It's my ray of hope: http://wingware.com/pipermail/wingide-users/2011-October/009381.html

And while I have your ear, I hope you don't mind me jumping in to this thread! I don't mean to step on toes, I just happen to bounce around between IDEs a lot, so I thought I'd offer what I could :)


Edit: Looking at Wingware's twitter, it looks like Wing 5 might not be too far off!
https://twitter.com/pythonide/status/312442932600897536

Message edited by Nicholas S. on 3/29/2013 - 12:49 PM

Hey Nicholas, finally got round to giving debugging in PyCharm a try but still no luck to be honest, don't know why. Following your instructions & entering the import etc into modo's interpreter just locks up modo, entering them into scripts just results in errors from the pydev module (bad file descriptor).
Cool! Thanks! I was wondering, is it possible to use Aptana for this as well?
Couldn't say I'm afraid, never used it.
Thanks for the IDE tests and detailed description.

I got PyCharm to work with Modo 701 on Mac OSX. But I got a few peculiar error messages that indicate to me that the python version for MAC OS wasn't compiled correctly as it references python files using windows path. Odd. I also don't understand why Modo used the rather old python version 2.6.

from pydev import pydevd
pydev debugger: CRITICAL WARNING: This version of python seems to be incorrectly compiled (internal generated filenames are not absolute)

pydevd.settrace('localhost', port=7720, stdoutToServer=True, stderrToServer=True, suspend=False)

pydev debugger: Unable to find real location for: C:\Python26\Lib\threading.py

However, the very small test program was debugged correctly


I'm thinking of giving this a try, but I was wondering if anyone had tried using Microsoft's PTVS with Modo. It seems to be working well with Maya. Just wondered if anyone had given this one a try... Thanks!

Sergio Mucino
TD/Rigger
www.maxtd.net

Haven't tried it I'm afraid.
Quote from Toonman :
I'm thinking of giving this a try, but I was wondering if anyone had tried using Microsoft's PTVS with Modo. It seems to be working well with Maya. Just wondered if anyone had given this one a try... Thanks!


Yes I did some testing with it. I made a few comments on it on this thread towards the bottom: http://forums.luxology.com/topic.aspx?mode=Post&f=37&t=75764&p=680992#680992

Here is a discussion I had with someone from Microsoft: https://pytools.codeplex.com/discussions/439880 Basically I was able to get it working, but without the ability to detach/reattach.

Ben
Quote from Matariki :
Thanks for the IDE tests and detailed description.

I got PyCharm to work with Modo 701 on Mac OSX. But I got a few peculiar error messages that indicate to me that the python version for MAC OS wasn't compiled correctly as it references python files using windows path. Odd. I also don't understand why Modo used the rather old python version 2.6.

from pydev import pydevd
pydev debugger: CRITICAL WARNING: This version of python seems to be incorrectly compiled (internal generated filenames are not absolute)

pydevd.settrace('localhost', port=7720, stdoutToServer=True, stderrToServer=True, suspend=False)

pydev debugger: Unable to find real location for: C:\Python26\Lib\threading.py

However, the very small test program was debugged correctly





I'm getting the same warnings. Did you ever find out what was causing it? I tried deleting pyc-files, but that didn't seem to work. Perhaps I need to reinstall python.

I'm using Aptana/pydev on ubuntu with 701sp3.
Is this possible with Wing IDE 101 (5.0.6)? 101 is working fine with my installed 3.4 and I'm self-teaching with my O'Reilly books but didn't find a *.py file in roaming to start mucking about with code that should really be tackled with a nice IDE buddy particularly since I have Python 3.4 on the system and Modo 701 is a confirmed to be 2.6.2. Tried all the other steps just for S&G knowing it wouldn't magically work so just wondering.

Oops, missed a page of instruction...
OK, put all the files in their places, changed flag as instructed, inserted definition in target *.py file. The file runs fine in modo, returns value as expected. But no response from Wing 101, no bug light, no option for passive listen. Will try again with Demo of Wing Standard next. Haven't graduated to plugin coding yet, still need to find documents on what the differences are aside from obvious ability to integrate into modo ui.

Wing Personal 5.0.6 works fine testing with rdebugtest.py file per video. So no joy for Wing 101 it seems.

Message edited by stephanpark on 5/18/2014 - 7:14 PM

Page 1 of 2
   1  2  Next  Last