[of-dev] windows 0071 status

Matthew Gingold m at gingold.com.au
Mon May 14 10:07:58 PDT 2012


Hey Damian

That looks really cool...will definitely start using it - although for the
tests I did today I stuck with my previous method (I am already compiling
things about a zillion times, so wanted to keep the amount of re-compiling
and re-testing as low as possible)...

Ok so I did some tests of VS vs CB - from my last GH comment:

===============
At the request of Andrey Kamaev I've submitted a bug report to openCV at
http://code.opencv.org/issues/1932

I've now tested performance against VS openCV libs in develop (and I also
recompiled those libs with same settings as I used with mingw32)...

Obviously testing between VS and CB is a bit like comparing apples and
oranges - it's hard to tell if differences are due to gcc vs cl or whether
the differences in optimization account for performance gains/deficits.

Regardless on my machine I was seeing approximately a 4% average greater
efficiency in VS after around 70+K executions of cvAbsDiff, cvThreshold and
cvFindContours...so not too much to worry about perhaps?

I'd like to recompile libs with Cuda support before making a pull request...
===============

....and switch off JPEG, PNG, TIFF, support etc which I left on when
compiling the first time - I'm assuming we don't need these on?

Also when I recompiled VS openCV libs I got a slight increase in performance
which actually made VS about 2.8% faster, so around 6.5% faster than CB for
the opencvExample....of course this is all measured in microseconds per
execution so perhaps it's all a bit moot...?

Would love to stop compiling openCV ;-)

Below is a summary of target settings for final build on windows...please if
@arturoc, @theo and @zach let me know if I've missed or got something wrong.
And also if you want me to PR the VS libs as well...

Build
-----
BUILD_PACKAGE = ON (Not sure what it does but is defualt)
BUILD_WITH_DEBUG_INFO = ON
BUILD_WITH_STATIC_CRT = ON (CB) / OFF (VS - link errors if you leave this on
in MSVC, but seems ok for mingw32...anyone know what it does?)
All other BUILD = OFF

CMAKE
-----
CMAKE_BUILD_TYPE = Release
CMAKE_CXX_FLAGS_RELEASE = -O1 -DNDEBUG

ENABLE
------
ENABLE_SSE = ON
ENABLE_SSE2 = ON
All other ENABLE = OFF

USE
---
USE_FAST_MATH = OFF (seems debatable whether to have on or off - could
someone please give me the official word?)
USE_OMIT_FRAME_POINTER = ON
USE_PRECOMPILED_HEADERS = ON

WITH
----
WITH_CUDA = ON (@arturo can you tell me which Cuda SDK version you are
using? - I think 2.3.1 only works with 4.0, yes? I fail with 4.2 already,
trying 4.0 now...if it's not essential I'll leave this out, but I think some
people are using this?)
WITH_EIGEN = ON
All other WITH = OFF


On 13/05/12 11:09 AM, "Damian Stewart" <damian at frey.co.nz> wrote:

> if we're talking timing things here, may i politely point you all at my addon
> ofxProfile:
> 
> https://github.com/damiannz/ofxProfile
> 
> :-)
> 
> cheers
> d
> --
> damian stewart . creative code
> http://damianstewart.com . twitter @damian0815
> 
> 
> 
> 
> On 13.05.2012, at 08:38, Elliot Woods wrote:
> 
>> @arturo - SSE is 128 bit not byte, correct? i.e. exactly 4 floats
>> so the fastest speedup would be 16x speedup (presuming you're operating on a
>> 1 byte grayscale image L8)
>> 
>> @matthew - maybe worth giving a time in VS for O3 w/SSE to see what we're
>> missing out on?
>> i.e. how much slower is it because of this bug vs properly working
>> 
>> On 13 May 2012 01:05, Matthew Gingold <m at gingold.com.au> wrote:
>> Great!
>> 
>> For ease of testing I make two branches with a modified opencvExample and
>> libs compiled as
>> 
>> 1) -O3 and SSE/SSE2 disabled:
>> https://github.com/gameoverhack/openFrameworks/tree/bugfix-openCV231_O3_NO_S
>> SE
>> 2) -O1 and SSE/SSE2 enabled:
>> https://github.com/gameoverhack/openFrameworks/tree/bugfix-openCV231_O1_WITH
>> _SSE
>> 
>> 
>> On 12/05/12 4:10 PM, "Damian Stewart" <damian at frey.co.nz> wrote:
>> 
>>>> @arturo: I am testing on a real windows install (win 7, core 2 duo). I too
>>>> thought compiling without SSE/SSE2 would cause major slow-downs, hence
>>>> persisting with compilation. Perhaps could you try the libs I compiled with
>>>> SSE/SSE2 enabled with -O1? And see if you think they are fast enough?
>>> 
>>> i'm back in my studio in vienna on Monday and can test a real windows 7 boot
>>> then...
>>> 
>>> cheers
>>> d
>>> --
>>> damian stewart . creative code
>>> http://damianstewart.com . twitter @damian0815
>>> 
>>> 
>>> 
>>> 
>>> On 12.05.2012, at 14:00, Matthew Gingold wrote:
>>> 
>>>> Hey arturo and elliot
>>>> 
>>>> @eliot: I did not try dynamic libs, tried many combinations with
>>>> static...:-)
>>>> https://github.com/gameoverhack/openframeworks/tree/bugfix-openCV231
>>>> 
>>>> How much does -O1 vs -O3 effect speed? Certainly it changes size of files,
>>>> but I do not know how to test whether -O1 is less efficient than -O3...
>>>> 
>>>> M
>>>> 
>>>> On 12/05/12 2:54 PM, "arturo castro" <arturo at openframeworks.cc> wrote:
>>>> 
>>>>> i'm testing this and also having problems with sse, it's weird since the
>>>>> code for that function doesn't have anything specific for windows so it
>>>>> should be the same in linux and osx, and at least in linux sse and sse2
>>>>> are enabled and they are no problem. it could be a bug in mingw though.
>>>>> 
>>>>> disabling sse means no simd, which means really serious slow downs.
>>>>> 
>>>>> btw has anyone tried this in a real windows machine, ie: no virtual
>>>>> machine? a virtual machine with vt enabled could be reporting sse but
>>>>> then doing some weird things when actually trying to use sse calls
>>>>> 
>>>>> El 12/05/12 14:04, Elliot Woods escribió:
>>>>>> did you try both static + dynamic / see if there was a difference in
>>>>>> reliability there?
>>>>>> 
>>>>>> On 11 May 2012 19:20, Theodore Watson <theo at openframeworks.cc
>>>>>> <mailto:theo at openframeworks.cc>> wrote:
>>>>>> 
>>>>>>    Great!
>>>>>>    Yeah I have had the sse issue with a lot of random libs / projects in
>>>>>> CB
>>>>>> 
>>>>>>    We should add a bit note about that to:
>>>>>> 
>>>>>> https://github.com/openframeworks/openFrameworks/wiki/Building-Libraries-
>>>>>> fo
>>>>>> r-
>>>>>> OF
>>>>>> 
>>>>>>    ----------------------------------------------
>>>>>>    Theo Watson
>>>>>>    http://theowatson.com
>>>>>>    http://openframeworks.cc
>>>>>>    ----------------------------------------------
>>>>>> 
>>>>>>    On May 11, 2012, at 2:06 PM, Matthew Gingold wrote:
>>>>>> 
>>>>>>> Hey Theo
>>>>>>> 
>>>>>>> I'll make a GH issue in a sec. I think your comments answer the
>>>>>>    problem. I
>>>>>>> tested very thoroughly different compile options for 2.3.0, 2.3.1
>>>>>>    and 2.4.0
>>>>>>> each time only against opencvExample and opencvHaarFinderExample.
>>>>>>> 
>>>>>>> I used latest distro's from openCV sourceforge (not SVN), and
>>>>>>    made sure to
>>>>>>> rebuild projects with PG and clean/build every time.
>>>>>>> 
>>>>>>> My mistake: I assumed that we used SSE/SSE2 optimizations. These
>>>>>>    are the
>>>>>>> problem. Current libs in Develop must have been compiled with
>>>>>>    these switched
>>>>>>> on (openCV 2.3.1 distro defaults to these on - same reason I left
>>>>>>    them on
>>>>>>> for a long time until this afternoon).
>>>>>>> 
>>>>>>> To summarise:
>>>>>>> 
>>>>>>> 1. With openCV compiled with SSE/SSE2 on, release and -O3 there
>>>>>>    IS problems
>>>>>>> - eg., cvThreshold and cvFindContours crash with SIGSEGV. Current
>>>>>>    openCV
>>>>>>> libs in develop seem to be compiled with these settings as they
>>>>>>    exhibit same
>>>>>>> crash pattern.
>>>>>>> 
>>>>>>> 2. With openCV compiled with SSE/SSE2 off, release and -O3 there
>>>>>>    is no
>>>>>>> problem.
>>>>>>> 3. With openCV compiled with SSE/SSE2 on, debug and -O3 there is
>>>>>>    no problem.
>>>>>>> 4. With openCV compiled with SSE/SSE2 on, release and -O1 there is no
>>>>>>> problem.
>>>>>>> 
>>>>>>> Libs in my branch are compiled as per 4, but it is easy to
>>>>>>    replace them with
>>>>>>> a version compiled as per 2, above (I just left the office but
>>>>>>    can upload
>>>>>>> them over the weekend)
>>>>>>> 
>>>>>>> I don't think the issue was with openMP or ofxFerns etc...just
>>>>>>    the SSE/SSE2
>>>>>>> optimizations. 100% sure this is the issue.
>>>>>>> 
>>>>>>> M
>>>>>>> 
>>>>>>> 
>>>>>>> On 11/05/12 7:26 PM, "Theodore Watson" <theo at openframeworks.cc>
>>>>>>    wrote:
>>>>>>> 
>>>>>>>> Hi Matthew,
>>>>>>>> 
>>>>>>>> I'm just looking at this for the first time.
>>>>>>>> Can you make a github issue about this, with as much info as you
>>>>>>    can - as I
>>>>>>>> think it would be good to have on GH.
>>>>>>>> 
>>>>>>>> I have had some similar experiences with opencv threading issues
>>>>>>    in CB and
>>>>>>>> also optimization issues in CB.
>>>>>>>> 
>>>>>>>> 1) Don't use any SSE optimizations. Also don't use fast math.
>>>>>>>> I've had crashing issues with ofxFerns and opencv which was
>>>>>>    directly related
>>>>>>>> to using both of those.
>>>>>>>> 
>>>>>>>> 2) Might be good to look at issues with OpenMP. For some reason
>>>>>>    that can cause
>>>>>>>> crazy issues like you're seeing.
>>>>>>>> See the entirety of this thread and especially this post:
>>>>>>>> 
>>>>>> 
>>>>>> 
http://forum.openframeworks.cc/index.php/topic,2994.msg16275.html#msg1627>>>>>>
5
>>>>>>>> Also check dlls aren't old ones.
>>>>>>>> 
>>>>>>>> Thanks!
>>>>>>>> Theo
>>>>>>>> 
>>>>>>>> ----------------------------------------------
>>>>>>>> Theo Watson
>>>>>>>> http://theowatson.com
>>>>>>>> http://openframeworks.cc
>>>>>>>> ----------------------------------------------
>>>>>>>> 
>>>>>>>> On May 11, 2012, at 1:06 PM, Matthew Gingold wrote:
>>>>>>>> 
>>>>>>>>> Ok I'll take a look at VS problems soon...
>>>>>>>>> 
>>>>>>>>> Been chasing the CB openCV lib problem...it's been super
>>>>>>    annoying (2 days of
>>>>>>>>> compile and re-compile), but I finally found some leads with:
>>>>>>>>> 
>>>>>>>>> http://code.opencv.org/issues/596
>>>>>>>>> and
>>>>>>>>> http://code.opencv.org/issues/1896
>>>>>>>>> 
>>>>>>>>> Unfortunately -O2 does not work for me...the most optimized I
>>>>>>    can get to is
>>>>>>>>> -O1. Id on't know too much about optimization settings - will
>>>>>>    that be ok for
>>>>>>>>> 0071?
>>>>>>>>> 
>>>>>>>>> I didn't apply the patch (cf issue 1896, above), but manually
>>>>>>    changed
>>>>>>>>> CMAKE_CXX_FLAGS_RELEASE "-O3 -DNDEBUG" to
>>>>>>    CMAKE_CXX_FLAGS_RELEASE "-O1
>>>>>>>>> -DNDEBUG" in CmakeCache before making the libs.
>>>>>>>>> 
>>>>>>>>> What's really annoying about this is that the bug is 1+ years
>>>>>>    old and also
>>>>>>>>> breaks 2.4.0 if compiled in Release with SSE/SSE2 support.
>>>>>>>>> 
>>>>>>>>> I could not get 2.4.0 to work with -O2 either, again I had to
>>>>>>    set to -O1.
>>>>>>>>> 
>>>>>>>>> Perhaps someone else would like to try? I'm using g++ version
>>>>>>    4.4.1 so maybe
>>>>>>>>> a more recent version would do a better job?
>>>>>>>>> 
>>>>>>>>> @Zach: did you make an issue for this problem on git? If so let
>>>>>>    me know so I
>>>>>>>>> can tag a PR with it...
>>>>>>>>> 
>>>>>>>>> You can try the libs I compiled at
>>>>>>>>> 
>>>>>>    https://github.com/gameoverhack/openframeworks/tree/bugfix-openCV231
>>>>>>>>> 
>>>>>>>>> M
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> On 11/05/12 3:01 PM, "Zachary Lieberman" <zach at eyebeam.org
>>>>>>    <mailto:zach at eyebeam.org>> wrote:
>>>>>>>>> 
>>>>>>>>>> I haven't had a chance to look too closely, but wrt to the
>>>>>>    debug vs
>>>>>>>>>> release libs, the project generator code here:
>>>>>>>>>> 
>>>>>>>>>> void visualStudioProject::addAddon(ofAddon & addon){
>>>>>>>>>> 
>>>>>>>>>> specifically:
>>>>>>>>>> 
>>>>>>>>>> found = addon.libs[i].find_last_of("\\")
>>>>>>>>>> 
>>>>>>>>>> looks very windows specific, and probably the logic is not the
>>>>>>    same on
>>>>>>>>>> non windows machines.   We should try to take a look more
>>>>>>    closely at
>>>>>>>>>> the logic, maybe test on linux and mac and see if there's
>>>>>>    something
>>>>>>>>>> either with path parsing or enumeration that's different.
>>>>>>>>>> 
>>>>>>>>>> take care,
>>>>>>>>>> zach
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On Thu, May 10, 2012 at 11:14 PM, Elliot Woods
>>>>>>>>>> <elliot at kimchiandchips.com <mailto:elliot at kimchiandchips.com>>
>>>>>>    wrote:
>>>>>>>>>>> Hmm seems super strange
>>>>>>>>>>> Working on other OS's with same compiler
>>>>>>>>>>> Working on same OS with different compiler
>>>>>>>>>>> :(
>>>>>>>>>>> 
>>>>>>>>>>> I'll try and have a look at this too (VS + CB).
>>>>>>>>>>> 
>>>>>>>>>>> On Friday, 11 May 2012, Matthew Gingold wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Re VS: I re-ran the PG before trying the
>>>>>>    project...unfortunately I just
>>>>>>>>>>>> left the office with the PC I've been using...perhaps could
>>>>>>    you try
>>>>>>>>>>>> rerunning the PG and see if it always does this with the
>>>>>>    latest dev? I'll
>>>>>>>>>>>> try in a couple of hours after some dinner...
>>>>>>>>>>>> 
>>>>>>>>>>>> As for the CB openCV problem the only thing I can think to
>>>>>>    do is try
>>>>>>>>>>>> compiling 2.3.0, 2.4.0 and maybe even 2.2.x just to
>>>>>>    eliminate any possible
>>>>>>>>>>>> blame with OF code and then take it to the openCV forums...?
>>>>>>>>>>>> 
>>>>>>>>>>>> Sent from my phone
>>>>>>>>>>>> 
>>>>>>>>>>>> On 10/05/2012, at 5:18 PM, Zachary Lieberman
>>>>>>    <zach at eyebeam.org <mailto:zach at eyebeam.org>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On the VS side, the linking looks odd for release:
>>>>>>>>>>>> 
>>>>>>>>>>>> debug is fine:
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
<AdditionalDependencies>%(AdditionalDependencies);opencv_highgui231d.lib;>>>>>>
o
>>>>>>>>>>>> pe
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
ncv_calib3d231d.lib;opencv_imgproc231d.lib;opencv_haartraining_engined.li>>>>>>
b
>>>>>>>>>>>> ;o
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
pencv_gpu231d.lib;opencv_flann231d.lib;opencv_contrib231d.lib;opencv_vide>>>>>>
o
>>>>>>>>>>>> 23
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
1d.lib;opencv_objdetect231d.lib;zlibd.lib;opencv_core231d.lib;opencv_ml23>>>>>>
1
>>>>>>>>>>>> d.
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
lib;opencv_features2d231d.lib;opencv_legacy231d.lib</AdditionalDependenci>>>>>>
e
>>>>>>>>>>>> s>
>>>>>>>>>>>> 
>>>>>>>>>>>> but the release project has both debug and release libs --
>>>>>>    maybe there's
>>>>>>>>>>>> some logic that needs to be fixed in the PG wrt figuring out the
>>>>>>>>>>>> difference
>>>>>>>>>>>> between debug and release libs?
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
<AdditionalDependencies>%(AdditionalDependencies);opencv_highgui231d.lib;>>>>>>
o
>>>>>>>>>>>> pe
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
ncv_calib3d231.lib;opencv_imgproc231d.lib;opencv_haartraining_engined.lib>>>>>>
;
>>>>>>>>>>>> op
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
encv_gpu231d.lib;opencv_flann231.lib;opencv_contrib231d.lib;opencv_video2>>>>>>
3
>>>>>>>>>>>> 1d
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
.lib;opencv_objdetect231d.lib;zlib.lib;opencv_core231d.lib;opencv_contrib>>>>>>
2
>>>>>>>>>>>> 31
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
.lib;opencv_ml231d.lib;opencv_features2d231.lib;opencv_core231.lib;opencv>>>>>>
_
>>>>>>>>>>>> gp
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
u231.lib;opencv_legacy231d.lib;opencv_haartraining_engine.lib;opencv_high>>>>>>
g
>>>>>>>>>>>> ui
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
231.lib;opencv_ml231.lib;opencv_imgproc231.lib;opencv_objdetect231.lib;op>>>>>>
e
>>>>>>>>>>>> nc
>>>>>>>>>>>> v_legacy231.lib;opencv_video231.lib</AdditionalDependencies>
>>>>>>>>>>>> 
>>>>>>>>>>>> This was in the project that was generated by arturo in the
>>>>>>    packages he
>>>>>>>>>>>> put up...  I'll check the develop branch, maybe something in
>>>>>>    how the files
>>>>>>>>>>>> are enumerated, etc are different and the logic breaks down?
>>>>>>>>>>>> 
>>>>>>>>>>>> thanks,
>>>>>>>>>>>> zach
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On Thu, May 10, 2012 at 11:13 AM, Matthew Gingold
>>>>>>    <m at gingold.com.au <mailto:m at gingold.com.au>>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> Ok I¹ve been looking into:
>>>>>>>>>>>> 
>>>>>>>>>>>> a) openCV VS issues ­ I¹m not having this problem...are you
>>>>>>    still seeing
>>>>>>>>>>>> this Zach? With both openCvExample and haar? Or just one of
>>>>>>    them?
>>>>>>>>>>>> 
>>>>>>>>>>>> And
>>>>>>>>>>>> 
>>>>>>>>>>>> a) openCV CB issues ‹ ok this is really frustrating. I¹ve
>>>>>>    recompiled
>>>>>>>>>>>> openCV 2.3.1 as static release libs. The problem persists ­
>>>>>>    still getting
>>>>>>>>>>>> immediate crash with openCvExample in both debug and release
>>>>>>    modes.
>>>>>>>>>>>> Uber-confoundingly after compiling debug static libs and
>>>>>>    linking against
>>>>>>>>>>>> those the example runs fine in both release and debug
>>>>>>    modes....so...I¹m
>>>>>>>>>>>> now
>>>>>>>>>>>> stumped ­ the problem seems to be actually in the libs
>>>>>>    themselves unless
>>>>>>>>>>>> I¹m
>>>>>>>>>>>> missing something completely...it is very consistently with
>>>>>>    cvThreshold
>>>>>>>>>>>> command, although sometimes cvFindContours crashes sigsegv
>>>>>>    too...should I
>>>>>>>>>>>> try 2.3.0 or 2.4.0 to confirm it¹s a specific problem with
>>>>>>    2.3.1 ? Any
>>>>>>>>>>>> common flags we use for oF would good to know about...right
>>>>>>    now I¹m
>>>>>>>>>>>> compiling with standard/defaults...
>>>>>>>>>>>> 
>>>>>>>>>>>> M
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> On 10/05/12 2:01 AM, "Zachary Lieberman" <zach at eyebeam.org
>>>>>>    <mailto:zach at eyebeam.org>> wrote:
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> 
>>>>>>>>>>>> here's a hit list for what needs to be done for 0071 / windows.
>>>>>>>>>>>> Matthew and Elliot,  (and anyone else windows based)  do you
>>>>>>    think you can
>>>>>>>>>>>> take a look?     We're almost there but switching the libs
>>>>>>    always leads to
>>>>>>>>>>>> stuff like this.
>>>>>>>>>>>> 
>>>>>>>>>>>> + + + + + +
>>>>>>>>>>>> VS
>>>>>>>>>>>> + + + + + +
>>>>>>>>>>>> 
>>>>>>>>>>>> a) we have some linking errors with the new opencv VS libs
>>>>>>    -- the debug
>>>>>>>>>>>> projects that use opencv are unhappy and don't compile:
>>>>>>>>>>>> 
>>>>>>>>>>>> opencv_core231d.lib(lapack.obj) : error LNK2038: mismatch
>>>>>>    detected for
>>>>>>>>>>>> '_ITERATOR_DEBUG_LEVEL': value '2' doesn't match value '0'
>>>>>>    in main.obj
>>>>>>>>>>>> 
>>>>>>>>>>>> b) release versions of all OSC examples are all failing to build
>>>>>>>>>>>> 
>>>>>>>>>>>> c) GPUparticleSystemExample fails to build (debug and release)
>>>>>>>>>>>> 
>>>>>>>>>>>> error C2668: 'sqrt' : ambiguous call to overloaded function
>>>>>>    [C:\Documents
>>>>>>>>>>>> and Settings\Administrator\My
>>>>>>>>>>>> 
>>>>>> 
>>>>>> 
Documents\0071\of_preRelease_v0071_vs2010\examples\gl\GPUparticleSystemEx>>>>>>
a
>>>>>>>>>>>> mp
>>>>>>>>>>>> le\GPUparticleSystemExample.vcxproj]
>>>>>>>>>>>>         C:\Program Files\Mic
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> --
>>>>>>>>>>> --
>>>>>>>>>>> Elliot Woods
>>>>>>>>>>> elliot at KimchiAndChips.com
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> of-dev mailing list
>>>>>>>>>>> of-dev at dev.openframeworks.cc
>>>>>>    <mailto:of-dev at dev.openframeworks.cc>
>>>>>>>>>>> 
>>>>>>    http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
>>>>>>>>>>> 
>>>>>>>>>> _______________________________________________
>>>>>>>>>> of-dev mailing list
>>>>>>>>>> of-dev at dev.openframeworks.cc <mailto:of-dev at dev.openframeworks.cc>
>>>>>>>>>> http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> _______________________________________________
>>>>>>>>> of-dev mailing list
>>>>>>>>> of-dev at dev.openframeworks.cc <mailto:of-dev at dev.openframeworks.cc>
>>>>>>>>> http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
>>>>>>>> 
>>>>>>>> _______________________________________________
>>>>>>>> of-dev mailing list
>>>>>>>> of-dev at dev.openframeworks.cc <mailto:of-dev at dev.openframeworks.cc>
>>>>>>>> http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
>>>>>>> 
>>>>>>> 
>>>>>>> _______________________________________________
>>>>>>> of-dev mailing list
>>>>>>> of-dev at dev.openframeworks.cc <mailto:of-dev at dev.openframeworks.cc>
>>>>>>> http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
>>>>>> 
>>>>>>    _______________________________________________
>>>>>>    of-dev mailing list
>>>>>>    of-dev at dev.openframeworks.cc <mailto:of-dev at dev.openframeworks.cc>
>>>>>>    http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> --
>>>>>> Elliot Woods
>>>>>> elliot <mailto:elliot at kimchiandchips.com>@KimchiAndChips.com
>>>>>> <http://www.kimchiandchips.com/>
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _______________________________________________
>>>>>> of-dev mailing list
>>>>>> of-dev at dev.openframeworks.cc
>>>>>> http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
>>>>> 
>>>>> _______________________________________________
>>>>> of-dev mailing list
>>>>> of-dev at dev.openframeworks.cc
>>>>> http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
>>>> 
>>>> 
>>>> _______________________________________________
>>>> of-dev mailing list
>>>> of-dev at dev.openframeworks.cc
>>>> http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
>>> 
>>> _______________________________________________
>>> of-dev mailing list
>>> of-dev at dev.openframeworks.cc
>>> http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
>> 
>> 
>> _______________________________________________
>> of-dev mailing list
>> of-dev at dev.openframeworks.cc
>> http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
>> 
>> 
>> 
>> -- 
>> -- 
>> Elliot Woods
>> elliot at KimchiAndChips.com
>> 
>> 
>> _______________________________________________
>> of-dev mailing list
>> of-dev at dev.openframeworks.cc
>> http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc
> 
> _______________________________________________
> of-dev mailing list
> of-dev at dev.openframeworks.cc
> http://dev.openframeworks.cc/listinfo.cgi/of-dev-openframeworks.cc





More information about the of-dev mailing list