Multiple projections on actors vrml-hair, vrml-hat, shadow-video

MOA (Making of Americans)

Workshop Production Report:
In 2002 Gertrude Stein Rep was able to spend a semester (January 2002 - May 2002) working on “The Silent Scream of Martha Hersland” Part two of a series of plays base on Gertude Stein’s novel “Making of Americans” at the University of Iowa. This report is a look back on the technical results of that workshop.
It is not a reflection on the full production which was devised to facilitate development of the script, staging, and process as well as technical needs.
This technical post mortum turned out to be the starting point of the creation of the hardware/software solution called “MediaBeam”. The MOA production was our first use of that type of system, and impetus for further development.

The features and ease of use of the “MediaBeam” and other robotic projection system has increased and changed over the last few years, and a follow up report in similar form on the MediaBeam will soon follow. It is always the look forward that makes the look back interesting.

Assessment of Assumptions:
_______________________________________________
Mirrors are a good possibly best way of moving a projected image about the stage Space.

Areas of weekness:

• Loss of brightness from mirror reflectivity (minor)
• Increased keystone and rotation (major issue)
• Loss of brightness due to rotation corrections
• Limited range of motion

Areas of strength:

• Easily adaptable to multiple projectors
• Very inexpensive “Off the shelf” theatrical products available I-cue $600 Elipscan $300)

Paths to improvements:

• mirror heads with Improved motor speed interface
• mirror heads with Improved accuracy
• mirror heads with Improved range of motion
  Software could improve motor speed and compensate for much of the accuracy
• Design custom mirror (dual mirror perhaps) that countered rotation effects
• Design of mirror with better speed interface and motion damping.

Alternatives:

Move the whole projector with larger motors.
Projectors are becoming smaller and some reasonably affordable off the shelf solutions for moving projectors are available though cost is higher.

Move the projector by hand (like follow-spot)
Uses a lot of manpower, and removes integration of cues with media.
However it is far more adaptable to changes on stage, and requires less software.

_______________________________________________
DMX as projector control

Areas of weekness:

• Cost compared to rs232 serial interface
• Lack of detailed knowledge of spec
  (Timing delays are they inherent or due to software)
• Belief that everything should run on Cat5 at least, probably over Ethernet

Areas of strength:

• Quick solution
• Off the shelf and theatre standard (I-cue and Elipscan)
• Integrate with lighting control possible
• Cost not high when large number of mirrors used, or when used for lighting control as well as mirror control (shared lanbox, and shared power supply)

Areas of weekness:

• Cost for small number of mirrors high. ($400 for lanbox, and $300 for scroller power supply needed by I-cue)
• Not commonly used for motion control outside intelligent lights
• realistically needs separate control hardware and interface than lights.
• off the shelf mirrors limited in size (note $10,000 large mirror available but not reasonable)

Paths to improvements:

Improve software latency
Integrate with lighting control
Make it a “plug able solution architecture” serial, DMX, midi
Increase ease of integration and possibly cost by using midi to interface with external lighting boards.
Identify and accommodate DMX timing issues

Alternatives:

• RS232
• Midi
• No robotics move projector by hand.

_______________________________________________
Video or other imagery will need to be processed/corrected in software to accommodate movable images.

This seems to be a core enabling feature, which opens a large number of other abilities which are useful even without robotic mirror system.

Area of weakness:

• Requires large time input to create custom software
• Software correction decreases image quality compared to hardware (optical) correction.

Areas of Strength:

• Extremely flexible and adaptable
• Much cheaper than optics
• Reusable solution
• May be a commercially viable product.

Paths to improvement:

Add more features
Improve interface
Document and make a packaged “product”
Add color correction
Add sub image correction (smearing, stretching etc ala goo)
Add more 3d deformations for projecting onto curved surfaces.
Integrate directly with projectors or mirrors where there is a feedback loop where the mirror or projector reports it’s position and the software auto corrects.

Alternatives:

Optics: lenses or multiple mirror solutions
View camera type optics for projectors.
Use lots and lots of projectors

_______________________________________________
DirectX/D3D image correction

Areas of weakness:

• Difficult to create due to lack of knowledge
  Or perhaps because it is not a quick evolutionary/prototyping tool which is almost
  crucial in experimental usage
• Frame rate attained acceptable only on the fastest PC’s
  (Though looking forward this is not an issue due to increasing computer speed)

Areas of strength:

• Much “heavy lifting” code is done for you already
• Speed is acceptable, and hardware acceleration is inherently used
  Future speed increase is expected due to game development and hardware

Paths to improvements:

Faster PC’s (already here)
Better code due to increased familiarity with the technology
Improved flexibility and speed of software adaptation (don’t know if this is reasonable?)

Alternatives:

Other 3D environment (GL, game engines, VRML)
Utilize 2D image manipulation code.

Note: I am pursuing a VRML solution in parallel with the D3D development because due to increased PC speed and improved VRML browser the speed is acceptable (though not nearly as good as the C++ application) and the VRML application already provides many features that would need to be created in our C++ application. The primary reason for this development is that I am able to adapt and modify this solution in a very fluid way compared to a C++ application, so this may be a valid development solution, that we can then refactor as a improved C++ application once the needed parameters are determined.

_______________________________________________
Live Video Input is core Needed Feature

Valid Assumption: No

Areas of Weakness:

• Input is of limited (analog) resolution
• Input material is not directly associated with cues
• Use of non-computer controllable media and video processors requires additional manpower, and complexity of cues.

Areas of Strength:

• Flexible
• Ability to use recorded media through external playback devices
• Ability to use off stage video cameras as input
  (Though in the end this was not utilized)
• Lower CPU demands than video playback

Path to Improvements:

It is clear that the ability to play recorded video and multimedia in the image is crucial, and will aid in cueing.
However a desire for live input is also still clear since it allows integration with
external and in fact unknown media types.
We need the solution to play live video, recorded video, stills, flash animations,
and in a far thinking way, remote live video (e.g. video conferencing)
in addition the ability to use higher resolution live video (such as from screen
capture SVGA or higher resolution) is also desirable.

Alternatives:

As mentioned above the alternative is to concentrate on playback of recorded material. The D3D application will continue to get better at this, and the VRML solution is already better able to do this. In fact it is able to load stills, recorded video and flash movies. It’s ability to do live video input is dependent on loading a flash movie that is showing live video input in fact. Though inefficient this also gives the ability to use flash video conferencing for remote live video as well which could be a large plus considering GSRT’s past work with video conferencing. So far the VRML is ahead of the C++ application in this respect, and is the main way in which I find the solution is more quickly adaptable, since I have created a work alike with more features in considerably less work hours. This work alike suffers from somewhat less performance, and the fact that there are some built in limitations that will be reached. On the other hand the D3D C++ application will not have those limitations, but may not ever develop that far due to the time needed to develop the application.

_______________________________________________

Need Video Mixer for live mix and processing:

Valid Assumption: No (with qualifications)

• Since live action video was not used in this production mixer was not needed.
• The fact that no clear use for this effect became apparent does not indicate possible future use will not be found.
• In fact video processing, specifically simply switching and fading was actually crucial due to the fact that all recorded material was passed through the system as live video.

Areas of Weakness:

• Lack of technical cuew’s knowledge in this area
• Equipment not integrated into stored cues
• Equipment difficult to use, expensive and surprisingly difficult to use
  mostly due to cost limitations.
• Need more manpower to operate equipment than computerized effects.

Areas of Strength:

Not dependent on software, or computer speed
Theoretically more flexible for real-time operation.

Paths to Improvement:

Computer controlled equipment
Higher quality equipment (faders, switchers)
Acquiring more expensive but configurable equipment (matrix switcher)
Offloading work to software

Alternatives:

• Digital video effects
• Digital video switching (video server)

While we tried to use a computer aided effects switcher it’s complexity was high, and in effect it was a hardware switcher with a computer, not a system that could be used in custom development. The best option would be to add multiple video inputs and video effects to the Geometry correction software. Though this solution is limited by the computer system speed, computers will continue to get faster, and the best way to make the use of the effects viable is to allow them to be saved in the cues so that the show can be reliably reproduced. It also seems that the image manipulation inherent in cross fades and video effects is similar to the image manipulation used to correct the geometry, and a solution that included both would be useful in a larger number of situations.

Rehersal projecting on actors in white