Nathan Horne – Technical Artist

Coming 4/26!!

My last project, The Last of Us us getting Remastered release on PS4 with a special dose of HD All-The-Things !

Check out the info here: Playstation Blog

Couldn’t find any good examples of directly writing to the QImage bits without using numpy, so here’s a simple example 🙂 (PyQt)

This just accesses the image bits uchar* directly, and uses struct.pack to convert the qRgb values into bytes for the uchar * (via sip.voidptr)

</span>
<pre>import struct
from PyQt4 import QtGui

#initialize the QImage
img = QtGui.QImage(512, 512, QtGui.QImage.Format_RGB32)
#Build the color object up front, new QColor every pixel is SLOW!
color = QtGui.QColor()

#Initialize the sip uchar ptr
ucharptr = img.bits()
ucharptr.setsize(img.byteCount())

i = 0
w = float(img.width())
h = float(img.height())
halfw = w/2

for y in range(img.height()):
	for x in range(img.width()):
		hue = (y/h)*360
		if x<halfw:
			sat = (x/halfw)*255
			val = 255
		else:
			sat = 255
			val = 255 - (((x-halfw)/halfw)*255)

		#Set the color values as HSV, then convert to qRgb
		color.setHsv(hue, sat, val)

		#Pack the qRgb into bytes using struct
		#Each pixel is 4 bytes, write the slice
		ucharptr[i:i+4] = struct.pack('I', color.rgb())
		i+=4

img.save('c:/out.png')

Result should look like Maya’s spectrum color picker

(And yes, QImage.setPixel does work as an alternative, but speed is worse, and even the QImage documentation recommends avoiding it)

sip.wrapinstance returns the best possible matching class while shiboken.wrapInstance does not. This is a pain to deal with, because you have to know the class of the object before you get it…

Here’s my shiboken wrapInstance function that actually gives back the correct class instead (This function will allow you to wrapinstance for either PyQt or PySide and get the same result back).

import shiboken
from PySide import QtGui, QtCore

def wrapinstance(ptr, base=None):
	"""
	Utility to convert a pointer to a Qt class instance (PySide/PyQt compatible)

	:param ptr: Pointer to QObject in memory
	:type ptr: long or Swig instance
	:param base: (Optional) Base class to wrap with (Defaults to QObject, which should handle anything)
	:type base: QtGui.QWidget
	:return: QWidget or subclass instance
	:rtype: QtGui.QWidget
	"""
	if ptr is None:
		return None
	ptr = long(ptr) #Ensure type
	if globals().has_key('shiboken'):
		if base is None:
			qObj = shiboken.wrapInstance(long(ptr), QtCore.QObject)
			metaObj = qObj.metaObject()
			cls = metaObj.className()
			superCls = metaObj.superClass().className()
			if hasattr(QtGui, cls):
				base = getattr(QtGui, cls)
			elif hasattr(QtGui, superCls):
				base = getattr(QtGui, superCls)
			else:
				base = QtGui.QWidget
		return shiboken.wrapInstance(long(ptr), base)
	elif globals().has_key('sip'):
		base = QtCore.QObject
		return sip.wrapinstance(long(ptr), base)
	else:
		return None

First, I don’t want to go into why/when to use Singletons (Or even if they have a valid use in python)… And yes, they do tend to be confusing no matter how they are implemented.

After looking around a bit, I wasn’t happy with many of the ways of creating a “singleton” in python, they all had some major flaw (That affected how I wanted to use it anyways).

Here’s the ways that I found while researching and my issues with them..

Using __metaclass__:

class Singleton(type):
    def __init__(cls, name, bases, dict):
        super(Singleton, cls).__init__(name, bases, dict)
        cls.instance = None

    def __call__(cls,*args,**kw):
        if cls.instance is None:
            cls.instance = super(Singleton, cls).__call__(*args, **kw)
        return cls.instance

class MyClass(object):
    __metaclass__ = Singleton

My main problem with this method is that PyQt doesn’t allow for __metaclass__ because sip already defines a __metaclass__ for all PyQt objects. Otherwise this method seems fairly clean (Although metaclasses do seem to confuse many python users)

Overriding __new__:

class Singleton(object):
	_instance = None
	def __new__(cls, *args, **kwargs):
		if not cls._instance:
			cls._instance = super(Singleton, cls).__new__(
								cls, *args, **kwargs)
		return cls._instance

This one has a fairly glaring issue, and I don’t know why so many people recommend it.. This variant ends up with __init__ being called EVERY time you access it via Singleton(), which causes all sorts of issues. (You could track if __init__ has already run, but that’s even more crap your Singleton has to hack around in your class)

Creating a wrapper class (Too long to post here, here’s the recipe link):

http://code.activestate.com/recipes/52558-the-singleton-pattern-implemented-with-python/

this variant just seems messy to me, anytime __getattr__ and __setattr__ are being overwritten, there’s typically a better way (Not always of course).

Just raising an exception ( O.o ):

class Singleton:
	__single = None
	def __init__( self ):
		if Singleton.__single:
			raise Singleton.__single
		Singleton.__single = self

Well that’s just not what most people want… if you don’t have access to where the current instance is stored, you now have to understand how the singleton is implemented and access Singleton.__single in order to get it… Yuck.
It also means you have to check whenever you need it for an exception.

Overwriting itself and using __call__

class Singleton:
	def __call__(self):
		return self

Singleton = Singleton()

I like how simple this variant is, it certainly seems closer to what I want. The issue with this singleton is that it can’t be subclassed, but otherwise it seems like fairly elegant python.
(You could subclass by using Singleton.__class__ I suppose, but that means other people subclassing you have to be aware of it. And their class won’t automatically behave like a singleton without them also overwriting their own name as well)

My version (Overwrite itself only on __init__):

class Singleton(object):
	def __init__(self):
		globals()[self.__class__.__name__] = self

	def __call__(self):
		return self

Subclasses don’t have to know about how the singleton is implemented, as long as the call the parent __init__ they will be a singleton as well (Even in another module):

class SubSingleton(Singleton):
	def __init__(self):
		super(SubSingleton, self).__init__()

And there you have it (If you want it anyways…)
No issues with subclassing, since at declaration time (And all the way up until Singleton() is called) your object is still the class, not the instance.
init is only called once, as it should be. And no metaclass is used, so PyQt is happy.

References from my research:

http://stackoverflow.com/questions/42558/python-and-the-singleton-pattern

http://code.activestate.com/recipes/52558-the-singleton-pattern-implemented-with-python/

http://stackoverflow.com/questions/31875/is-there-a-simple-elegant-way-to-define-singletons-in-python

For Windows x64

PyQt 4.9 – Maya 2013-x64 
This is just a quick zip of all the files I am using, no installer. Just drop it into your Maya site-packages or site.addsitedir the unzipped folder.

Why use PyQt/PySide at all in Maya?

• A larger array of available GUI elements to work with.
• A visual editor(Qt Designer).
• Object Oriented design.
• Signal/Slot mechanism, combined with an event driven system.
• Access to low level components such as the base QWidget, QObject, QAction, and QPainter.
• Much more! We are going to go over many things that are simply impossible within the context of Maya’s default toolkit.

Some examples of cool things!

A proper tree widget, and full control over it.

Mounds of random GUI’s as you can see, icon support in PyQt is just about everywhere versus Maya UI where only a few specific controls support images (And never dynamic resizing of them).

PyQt/PySide, lets get this out of the way…

• PyQt and PySide are nearly identical as of right now. PySide is newer and is being sponsored and partially developed by the people who make Qt itself, Nokia.
• PyQt website (Made by Riverbank software)
• PySide website (Made by Nokia)
• For this article I will be covering primarily PyQt, but PySide is pretty much the same so just swap PyQt for PySide and you should be fine.
• PySide is available as LGPL as well as GPL. PyQt is GPL only.
• I will make a note if something is specifically different between the two.

Setting up PyQt

Hopefully by now you are saying “Gee, that sounds great, but how do I get it installed and working!”, fear no more! I am also going to go over some basic deployment information, in case you want to roll out PyQt to your team.

Setting up PyQt – Getting it!

Here are the versions of Maya, and their corresponding installers for PyQt. All have been tested in production env. and are verified working. Download the correct installer for your version of Maya below. (If you are on OS-X/Linux you will have to acquire these yourself sadly. If anyone wants to do a write up and provide links let me know.)  Credit goes to Robert Kist (http://blarg.robertkist.com/) for the 2012 X86 build. Thanks Robert!

• Maya 2011
• Maya 2011-x64
• Maya 2012
• Maya 2012-x64

Setting up PyQt – Installing it

I have always found that it’s easiest to install new packages into a clean python install before extracting them out into their own folder, or moving them into Maya’s site-packages directory. Here’s the installers for the versions of Python that Maya uses (And that the above PyQt installers will be looking for).
Maya2011/2012:
http://www.python.org/ftp/python/2.6.4/python-2.6.4.msi
Maya2011/2012 x64:
http://www.python.org/ftp/python/2.6.4/python-2.6.4.amd64.msi

Now install PyQt into that clean Python install.

Setting up PyQt – Making Maya aware of the PyQt and sip packages

There are multiple ways of doing this, and which works best depends on how your pipeline/deployment works. There are probably more ways as well, but these are the ones that I know of.

The last two of these, the .pth method and the site.addsitedir method are both great for deploying PyQt via a network location. With these you can just drop a single copy somewhere everyone has access to, and then during Maya startup (Or python in general with site-customize) add the correct path and voila PyQt is working.

Copying to Maya’s site packages

• Probably the easiest method for a single user who only wants to use PyQt from inside Maya.
• Has to be done per-user and involves having access to a copy of the files to copy over.
• Updates are somewhat difficult only because they need to be done per user (Unless you have a versioned Maya install)
• Copy the PyQt4 folder, and the sip.pyd file from your python26/lib/site-packages directory into the Maya install /python/lib/site-packages directory.

Creating a .pth file

• Still requires a file per-user, but not a full copy of the packages
• Updates are easy, just update the network/Perforced location
• Extremely simple file, and can be created after python startup but before pyqt import if need be.
• Create an empty text file in your Maya install /python/lib/site-packages directory and rename it to something like pyqt.pth
• Open that file in a text editor and paste in the full path to your PyQt4/sip site-packages folder ( python26/lib/site-packages)

site.addsitedir

• No files in the users Maya directory
• Requires code to be run before importing PyQt (Can easily be done in sitecustomize.py, or userSetup.py, or at head of script)
• Before trying to import PyQt or sip, use something like the following code
• import site
• site.addsitedir(r’PATH/TO/PYQT-SITE’)

Setting up PyQt – Test it!

In Maya, try the following code

import sip
sip.setapi('QString', 2)
sip.setapi('QVariant', 2)
from PyQt4 import QtGui, QtCore
btn = QtGui.QPushButton(QtGui.__file__)
btn.show()

If you don’t get any errors and a button is created (Not parented to Maya though) then it worked! Otherwise, leave me a comment with the error message from Maya, and I will try to help you out.

A quick note on sip

PySide uses the equivalent of the PyQt version 2 API, while PyQt has two versions that can be picked via sip.setapi(). To maintain code compatibility between the two and for the benefit of the improved API you will see “sip.setapi(‘blah’, 2) on occasion. sip only permits the API to be set before PyQt is loaded, so be aware that you cannot change API’s mid-run. Maya MUST be restarted for code using the alternate API to be run, and only if the api version is set properly before PyQt is imported. For this reason, try to pick an API version and stick with it (I highly recommend version 2…)

Diving in..

#Source code for some common Maya/PyQt functions we will be using
import sip
sip.setapi('QString', 2)
sip.setapi('QVariant', 2)
from PyQt4 import QtGui, QtCore
import maya.OpenMayaUI as apiUI

def getMayaWindow():
	"""
	Get the main Maya window as a QtGui.QMainWindow instance
	@return: QtGui.QMainWindow instance of the top level Maya windows
	"""
	ptr = apiUI.MQtUtil.mainWindow()
	if ptr is not None:
		return sip.wrapinstance(long(ptr), QtCore.QObject)

def toQtObject(mayaName):
	"""
	Convert a Maya ui path to a Qt object
	@param mayaName: Maya UI Path to convert (Ex: "scriptEditorPanel1Window|TearOffPane|scriptEditorPanel1|testButton" )
	@return: PyQt representation of that object
	"""
	ptr = apiUI.MQtUtil.findControl(mayaName)
	if ptr is None:
		ptr = apiUI.MQtUtil.findLayout(mayaName)
	if ptr is None:
		ptr = apiUI.MQtUtil.findMenuItem(mayaName)
	if ptr is not None:
		return sip.wrapinstance(long(ptr), QtCore.QObject)

PyQt and Maya – Parenting to the Maya Window

The process for parenting your window to the main Maya window is fairly simple. First you get the main Maya window as a QMainWindow instance using the getMayaWindow() function and then pass that instance in as the parent object for your new top level PyQt object.

PyQt and Maya – Parenting to a specific Maya Widget

Similar to the main Maya window, here the path for our Maya UI control is passed to the toQtObject() function to get a PyQt object. Next just parent the new object to the Maya PyQt one. The main issue here is going to be that Maya uses “invisible” layouts to control how objects are placed. I would generally recommend avoiding this use scenario unless absolutely necessary, navigating the Maya widget hierarchy can be a pain. Look for info on these invisible widgets in the Maya API docs for MScriptUtil, it shows how to query the internal Qt structure for the Maya widgets.

PyQt and Maya – Parenting a Maya widget to PyQt

First convert the Maya UI control name to a PyQt object then call .setParent() on that object and provide the new parent widget. You will probably need to assign it a layout in PyQt as well, depending on the structure of your UI.

PyQt and Maya – Customizing a Maya widget with PyQt

Convert your Maya UI control to a PyQt object. Customize as you normally would for that object type (Ex: assign a Qt style sheet, or add an image to a QPushButton)

PyQt general – Garbage collection

Python will garbage collect any locally scoped objects when a function has finished executing if their reference count drops to zero. The effect on us is that a window declared in a function will be immediately destroyed when that function ends, if that window is not stored elsewhere. The two most common ways to store it are either by declaring the window variable global, or by assigning it into the __main__ module (import __main__).

Qt Designer and you!

Why use Qt Designer?

• Esablishing layouts and getting everything properly positioned is more easily achieved visually (IMO)
• The designer files offload what is normally the largest portion of your code to a seperate file.
• XML is nice and easy to parse, and modify at run-time.
• It’s possible for non-programmers to tweak UI’s to some degree. (Customize icon sets, change tooltips, etc..)
• Utilize the Qt resource system (Although some modifications are necesarry at run-time or as a “build” step depending.

Pre-compiled designer files versus run-time

• Both PyQt and PySide have a utility for generating a python file based on the designer .ui file.
• Both PyQt and PySide have the ability to parse and convert the designer file to python objects on the fly.
• Unless you are having a huge hit startup time wise for your UI (Due to the parse step of the UI xml data) I would highly recommend converting on the fly.
• On the fly is much more dynamic, and you wont have to constantly maintain the intermediate .py file.
• Also there’s still an import step involved for the .py file, so it’s not going to save a substantial amount of time in most cases.
• If the UI file is load on the fly, it’s easy to modify the incoming xml file before loading it. This is most useful for finding resource paths and either compiling them, or handling them in a custom way.

How to load a designer file

• DO NOT USE cmds.loadUi!!!
• seriously, don’t.. forget it exists, now.
• Who ever wrote it needs to be punched in the mouth.
• Use uic/pysideuic (PySide has an alternate setup here), it’s awesome!
• Note: pysideuic is lacking a loadUiType function, I have a workaround that I will try to mention in the example code. (Edit: here’s some PySide loadUiType code)
• The uic.loadUiType function is the one that should be used for loading in a PyQt UI from a designer file.
• Depending on how your resources are set up you may wish to alter the UI file before passing it to the loadUiType command. (See resources section below)

from PyQt4 import uic
form_class, base_class = uic.loadUiType('path/to/uifile.ui')

The  form_class returned is your customized python class (Based on object) and base_class is the super class of the window (Ex: QMainWindow, QDialog, etc..). Now to begin using them simply subclass from both (multiple inheritance). In your __init__ method you need to use super() to call the proper parent __init__ functions, and then setupUi() on self to instantiate the designer widgets onto self. One important thing to note here, is that the form_class and base_class attributes need to be unique variables for each UI at the top level, or you will have issues when you go to create your window instances.

class Window(form_class, base_class):
    def __init__(self, parent=None):
        super(Window, self).__init__(parent)
        self.setupUi(self)

Here’s a more specific example showing how to use a designer file as a base, and extend it in code. We are going to create a simple list, where items can be added, and removed via a button. The layout portion of this is defined in Qt designer, which I’ve provided the file for (example1.ui file). Now let’s dig into some code! (Full source code)

#This relies on the earlier getMayaWindow() function, so make sure that this code has access to that.
from PyQt4 import uic

#If you put the .ui file for this example elsewhere, just change this path.
listExample_form, listExample_base = uic.loadUiType('c:/example1.ui')
class ListExample(listExample_form, listExample_base):
	def __init__(self, parent=getMayaWindow()):
		super(ListExample, self).__init__(parent)
		self.setupUi(self)

		#The names "addItemBtn" and "removeItemBtn"
		#come from the "objectName" attribute in Qt Designer
		#the attributes to access them are automatically created
		#for us when we call setupUi()
		#Designer ensures that the names are unique for us.
		self.addItemBtn.clicked.connect(self.addItem)
		self.removeItemBtn.clicked.connect(self.removeItem)

	def addItem(self):
		"""
		Add a new item to the end of the listWidget
		"""
		item = QtGui.QListWidgetItem(self.listWidget)
		item.setText('Item #%s!'%self.listWidget.count())

	def removeItem(self):
		"""
		Remove the last item from the listWidget
		"""
		count = self.listWidget.count()
		if count:
			self.listWidget.takeItem(count-1)

I have allot more planned, but Thanksgiving is coming up so I have decided to split this up into parts (probably at least 3, judging by how much of my overview I covered in this intro)

I’ve seen this question a few times now so here’s a really simple example of embedding a Maya widget into a PyQt UI

import maya.OpenMayaUI as apiUI
from PyQt4 import QtGui, QtCore
import sip

def getMayaWindow():
    ptr = apiUI.MQtUtil.mainWindow()
    return sip.wrapinstance(long(ptr), QtCore.QObject)

def toQtObject(mayaName):
    '''
    Given the name of a Maya UI element of any type,
    return the corresponding QWidget or QAction.
    If the object does not exist, returns None
    '''
    ptr = apiUI.MQtUtil.findControl(mayaName)
    if ptr is None:
        ptr = apiUI.MQtUtil.findLayout(mayaName)
    if ptr is None:
        ptr = apiUI.MQtUtil.findMenuItem(mayaName)
    if ptr is not None:
        return sip.wrapinstance(long(ptr), QtCore.QObject)

class MayaSubWindow(QtGui.QMainWindow):
    def __init__(self, parent=getMayaWindow()):
        super(MayaSubWindow, self).__init__(parent)
        self.executer = cmds.cmdScrollFieldExecuter(sourceType="python")
        qtObj = toQtObject(self.executer)
        #Fill the window, could use qtObj.setParent
        #and then add it to a layout.
        self.setCentralWidget(qtObj)

myWindow = MayaSubWindow()
myWindow.show()

So, we noticed some odd behavior with several threaded scripts at work: They ran slower than the initial single threaded version.. Off to the Maya docs!

Per the Maya docs on threading and python:

“One important difference between Python threads and C++-based threads is that python threads do not run concurrently since the Python interpreter itself is not currently threadsafe. For this reason, they are not useful for data parallel applications. However, they may be useful, for example, where polling of resources is done that might otherwise have to wait for timeouts.”

After doing a bit of research I thought I would share some of what I found on python and Threading, this sparked a few conversations at work about the usefulness and specific use cases of threads.

Short version:

• Maya Python and standalone Python handle threads differently: False
• Python instructions cannot run in parallel: True
• Python threads are not “true” threads: False
• Threads provide no speed benefit or parallelism: False
• Should you use threads speed up potentially parallel operations: Sometimes

Python’s natively limited via the Global Interpreter Lock (GIL) to prevent concurrent Python byte code instructions, this prevents separate threads from simultaneously modifying the same memory space in order to prevent  memory access issues. If one thread were able to delete a shared object while another thread tried to append to it, the result would be a crash without the GIL. On the internal implementation side it get’s even more important as internal data like weakref’s and reference counts could become out of sync.

The silver lining to this limitation is that it only effect’s pure python bytecode, anything happening via a compiled c extension or a linked C++ library like PyQt will generally try to release the GIL while the compiled code is running, as that code can’t modify the internal Python interpreter state anyways (C++ code couldn’t delete your python object for example). Most of the major and intensive operations that you would want to do in a thread are already careful to release the GIL when performing a blocking operation: file read/write, time.sleep, sql select/execute, xml parse, etc..

Per the python documentation on the GIL:

” The GIL is controversial because it prevents multithreaded CPython programs from taking full advantage of multiprocessor systems in certain situations. Note that potentially blocking or long-running operations, such as I/O, image processing, and NumPy number crunching, happen outside the GIL. Therefore it is only in multithreaded programs that spend a lot of time inside the GIL, interpreting CPython bytecode, that the GIL becomes a bottleneck.”

PyQt and the GIL:

“when the threaded code is calling long running c/c++ functions, such as a database select, it will release the GIL until the call completes.  This is all done automatically within python and PyQt.  This is “real” multithreading.”

As for the original issue raised by the Maya api docs, the real issue isn’t as straightforward as the docs would lead you to believe: Yes, using threads to number crunch in pure python threads is pointless, they will not run in parallel and will yeild no speed increase. However, file operations, numpy, sql access, and many other functions can be used safely within Maya and yield speed increases and truly parallel work. The docs seem to be primarily focused on parallelism in the context of deformers or number crunching/simulation plugins, in which case that computation should be offloaded to a custom c extension or something like numpy.

References:

• http://docs.python.org/glossary.html#term-global-interpreter-lock
• http://wiki.python.org/moin/GlobalInterpreterLock
• http://www.riverbankcomputing.co.uk/static/Docs/sip4/using.html
• http://jessenoller.com/2009/02/01/python-threads-and-the-global-interpreter-lock/http://www.riverbankcomputing.com/pipermail/pyqt/2011-January/028961.html
• http://download.autodesk.com/us/maya/2011help/index.html?url=./files/WS1a9193826455f5ff-7f10494411f03a24f24-7e6e.htm,topicNumber=d0e672317
• http://www.dabeaz.com/python/UnderstandingGIL.pdf