Wednesday, 6 December 2017

RBF Based Colour Reader

Years back I produced a node based colour reader that utilised the closest point utility node to read back texel values at a uv position. By supplying a ramp texture you could have the system spit out RGB values which being 0 to 1 based lend themselves perfectly to driving other systems. A drawback of this method was that the texture provided had to be procedurally generated. Textures created by hand in Photoshop would not work. This made it more difficult to customise the colours for a specific output. Another was that the type of surface it was limited to was NURBS.


However it did work well, especially on areas where setting an extreme position was not always simple. For example we have a couple of other pose space solutions available to us at our studio. One is the standard cone reader that takes the angle between two vectors to return a value based on how far that angle is between the centre and outer edge of a given radius.
The second takes a vector magnitude between a radial centre point and a target point and again tests if that magnitude sits inside or out of a given radius. These two different ways to return a value suffer from the same shortcoming. When the target point or angle passes into the given radius and hits 0 the output is maximum. Continue on though and the target passes the centre and travels back outside the radius. This leads to problems. For instance driving a blendshape with the output of either of these setups will result with the blendshape target climbing to full application before decreasing again. In areas such as shoulders on characters this can lead to unpredictable deformation unless many of these readers are utilised to counteract the problem.
Hitting and passing these extremes with the colour space reader results in the extreme value always reading maximum. This means that when a limb is pushed a bit too far the driven shapes do not start collapsing inwards again.

My colleagues and I have recently been investigating the application of RBF based solving in regards to large amounts of data. I decided it was time to rewrite the colour space reader, this time utilising the Maya API, C++ and the new RBF learnings.
So by using each mesh vertex on the reader as a ‘node point’ for the rbf and then throwing in the colours at each of these vertices as values it was possible to extrapolate a weighted result that could be output as a single rgba value. The beauty of this method is that rather than sampling texels which are more difficult to apply to the method the user can simply create any mesh, apply vertex colours as they wish to any of its vertices and get a result. Want to change the result? Change the colours or reshape the mesh. It’s nice and simple

The solution still uses closest point but through MFnMesh this time.
I have included a python version of this node with the post to get you started if you fancy a stab at it. This version does not use RBF instead weighting the colours based on distances.

RBF Based Colour Reader from SBGrover on Vimeo.

It’s worth noting that whilst creating this node i found an issue with Maya and the worldMesh output. If worldMesh is used then the colour output does not update when colours on the mesh change. This does not appears to happen with the python version but is worth keeping your eye on. If you find that you get this result you will need to adjust the node to use outMesh which will involve multiplying the inMesh by its own worldMatrix to convert the points to world space. You will also need to multiply the centre object MPoint by the inverse of this worldMatrix and use this new MPoint with the closestPoint calculation.

"""

import maya.cmds as mc
mc.delete(mc.ls(type='ColorSpaceReaderPy'))
mc.flushUndo()
mc.unloadPlugin('ColorSpaceReaderPy')
mc.loadPlugin('ColorSpaceReaderPy.py')
mc.createNode('ColorSpaceReaderPy')


mc.connectAttr('pPlaneShape1.worldMesh', 'ColorSpaceReaderPy1.inMesh', f=True)
mc.connectAttr('locator1.worldMatrix', 'ColorSpaceReaderPy1.centre', f=True)
mc.connectAttr('ColorSpaceReaderPy1.outClosest', 'pSphere1.translate', f=True)
mc.connectAttr('ColorSpaceReaderPy1.outColor', 'lambert2.color', f=True)

"""

import maya.OpenMaya as om
import maya.OpenMayaMPx as omMPx

kPluginNodeTypeName = "ColorSpaceReaderPy"
kPluginNodeClassify = 'utility/general'
kPluginNodeId = om.MTypeId(0x81012)

class ColorSpaceReader(omMPx.MPxNode):

 inMesh = om.MObject()
 inCentre = om.MObject()
 outClosest = om.MObject()
 outColor = om.MObject()
 output = om.MObject()

 def __init__(self):

  omMPx.MPxNode.__init__(self)

 def compute(self, plug, data):

  inMeshData = data.inputValue(ColorSpaceReader.inMesh).asMesh()
  inCentreMatrix = data.inputValue(ColorSpaceReader.inCentre).asMatrix()
  outColorHandle = data.outputValue(ColorSpaceReader.outColor)
  outClosestHandle = data.outputValue(ColorSpaceReader.outClosest)

  if not inMeshData.isNull():
   meshFn = om.MFnMesh(inMeshData)
   sourceVerts = om.MPointArray()
   colors = om.MColorArray()
   meshFn.getPoints(sourceVerts, om.MSpace.kWorld)
   meshFn.getVertexColors(colors)
   centrePos = om.MPoint(inCentreMatrix(3, 0), inCentreMatrix(3, 1), inCentreMatrix(3, 2))
   closestPoint = om.MPoint()
   mu = om.MScriptUtil()
   mu.createFromInt(0)
   polygon = mu.asIntPtr()
   meshFn.getClosestPoint(centrePos, closestPoint, om.MSpace.kWorld, polygon)
   closestPolygon = mu.getInt(polygon)

   faceVertArray = om.MIntArray()
   meshFn.getPolygonVertices(closestPolygon, faceVertArray)

   colorArray = om.MColorArray()
   magArray = om.MFloatArray()

   for i in range(faceVertArray.length()):
    mag = closestPoint.distanceTo(sourceVerts[faceVertArray[i]])
    magArray.append(mag);
    colorArray.append(colors[faceVertArray[i]])

   weights = om.MFloatArray(magArray.length(), 0.0)
   foundOne = 0
   weightsTotal = 0.0

   for i in range(magArray.length()):

    if magArray[i] == 0.0:
     weights.set(1.0, i)
     foundOne = 1
     weightsTotal = 1

   if foundOne == 0:

    for i in range(magArray.length()):
     weights.set(1.0 / magArray[i], i)
     weightsTotal += weights[i]

   unit = 1.0 / weightsTotal
   weightedColor = [0, 0, 0]

   for i in range(magArray.length()):
    w = unit * weights[i]
    weightedColor[0] += colorArray[i][0] * w
    weightedColor[1] += colorArray[i][1] * w
    weightedColor[2] += colorArray[i][2] * w

   weightedColorVec = om.MFloatVector(weightedColor[0], weightedColor[1], weightedColor[2])
   outClosestHandle.setMFloatVector(om.MFloatVector(closestPoint))
   outColorHandle.setMFloatVector(weightedColorVec)


def nodeCreator():

 return omMPx.asMPxPtr(ColorSpaceReader())
 
def nodeInitializer():
 nAttr = om.MFnNumericAttribute()
 mAttr = om.MFnMatrixAttribute()
 tAttr = om.MFnTypedAttribute()

 ColorSpaceReader.inMesh = tAttr.create("inMesh", "im", om.MFnData.kMesh)
 tAttr.setReadable(0)
 tAttr.setKeyable(1)

 ColorSpaceReader.inCentre = mAttr.create("centre", "c")
 tAttr.setReadable(0)
 tAttr.setKeyable(1)

 ColorSpaceReader.outClosest = nAttr.createPoint("closestPoint", "cp")
 tAttr.setReadable(1)
 tAttr.setKeyable(0)

 ColorSpaceReader.outColor = nAttr.createPoint("outColor", "col")
 tAttr.setReadable(1)
 tAttr.setKeyable(0)

 ColorSpaceReader.addAttribute(ColorSpaceReader.inMesh)
 ColorSpaceReader.addAttribute(ColorSpaceReader.inCentre)
 ColorSpaceReader.addAttribute(ColorSpaceReader.outClosest)
 ColorSpaceReader.addAttribute(ColorSpaceReader.outColor)

 ColorSpaceReader.attributeAffects(ColorSpaceReader.inMesh, ColorSpaceReader.outColor)
 ColorSpaceReader.attributeAffects(ColorSpaceReader.inCentre, ColorSpaceReader.outColor)
 ColorSpaceReader.attributeAffects(ColorSpaceReader.inMesh, ColorSpaceReader.outClosest)
 ColorSpaceReader.attributeAffects(ColorSpaceReader.inCentre, ColorSpaceReader.outClosest)

def initializePlugin(mobject):
 fnPlugin = omMPx.MFnPlugin(mobject)
 fnPlugin.registerNode(kPluginNodeTypeName,kPluginNodeId,nodeCreator,nodeInitializer,omMPx.MPxNode.kDependNode,kPluginNodeClassify)
 
def uninitializePlugin(mobject):
 fnPlugin = 
omMPx.MFnPlugin(mobject)
 fnPlugin.deregisterNode(kPluginNodeId)

0 comments:

Post a Comment