Gloria Stern talks with Jeff Green
By Gloria Stern

Jeff first came to my attention as the writer of "Midnight Stranger," which incorporated some highly original interactive techniques. As hackneyed and trite as the word is, I have to use the label "pioneer" when talking about him. Jeff has held a variety of jobs in a variety of media. After our virtual meeting, I became a fan. As he talked about filming, facilitating interactives and philosophy, I became an advocate-- nay, a Jeff Green groupie...

GS: The first thing I noticed about your personal history is that you have had quite a number of careers. What brought you to interactive media?

JG: I've always considered my 'career' to be somehow independent of whatever medium I was able to work in, though I know that flies in the face of traditional approaches to an individual's life and work. Our society loves to label people as early as possible, as a matter of convenience, a kind of file and forget mentality.

My interests have been in: media in general, immersive storytelling, the simulation of reality, the stimulation of mind, the shamanistic power of technology/art/spirit to transform the landscape of the species... as you can see I don't have an easily definable focus and so any media can serve to further the cause.

Basically I have tried to do whatever was possible in whatever form came to hand. When all I had was a typewriter, I wrote. When I got a tape recorder I made stories in sound. If this had been ten years later I might have got a video camera as a child and my story would be different, but for the longest time sound was the most obvious and accessible outlet and that led me through college radio and into professional, where my first story cycle was created. A contact through the radio work got me into my television series, and an effort to produce my own computer animation for that series led me to Animatics where the idea for the interactive stuff was hatched. Not exactly a textbook pattern, I know...

Also in there I spent time as, amongst other things, a veejay, journalist and canoe instructor...

GS: Rather an unusual schooling for what you are doing now. What in your background qualified you for developing interactive fiction?

JG: The impetus to push the limits of available media was always there, and the experimentations I undertook often included interactive elements. Some of my earliest written work was based on random exchanges with unsuspecting subjects. For years I produced large-scale installation parties with multiple screens and ongoing audience feedback. For some time I unsuccesfully pursued ways that I could adapt my radio drama to the CD-ROM medium, adding interactivity and graphical elements. In addition I was interested in computers before it became fashionable, undertaking courses in animation and electronic music when those technologies were still huge and awkward. So when the Animatics opportunity arose I had very little difficulty in getting my head around the potential of the form.

GS: When you began Midnight Stranger, it definitely broke new ground. What did you have to work with at the onset?

JG: I cannot claim the original idea for MS, which credit rightly belongs to Simon Goodwin, co- founder of Animatics. The discussions we first undertook there had to do with an idea of Simon's for a social simulator that really grew out of his fascination with the kind of conversational interplay that can take place in the context of urban nightlife. This concept of playing head games with party people came along with an interaction device that was like a control panel at the bottom of the screen that allowed you to input your side of the conversation as an abstracted emotion. Over the course of several discussions Alfredo and I helped develop the idea further.

They wanted an entertainment product and so felt it needed some kind of traditional scripting, and they wanted me to take what was still basically an amorphous notion with a few technical directions established and generate a story on which it could hang. I started out creating some elaborate multi-pathway scenarios, but as it turned out I ended up getting much more drawn to the reality simulation aspects of the production, and downplayed the story elements dramatically.

GS: Starting from the available technology, eh? What were the limitations?

JG: Legion. First of all, we had no money. Animatics was still a young company so there was some equipment but not a lot of free capital. We all donated our spare time to the demo, and the cast for that were pretty well all friends.

Next, I was shocked by the limitations of the technology, coming as I was from television, which has an apparently limitless memory capacity and constant full-screen motion. To be presented with the realities of using video in a CD-ROM was a rude awakening.

These were the days before people were even talking about multiple-disc sets, so we only ever thought about the single CD. In order to get anything like what I considered a minimum amount of material on the disc to make it a valid product, at a decent level of image and sound quality, we had to reduce the moving image size to a small fraction of the screen. I found this unacceptable, and so proposed the frame-in-frame approach as a compromise, embedding the movie in a still image with an attempt made to register the two.

Other limitations, when it came to the actual shooting, were also related to budget, but may have added to the atmosphere of the final product. The crew was very small -- sometimes just me with a camera and lighting kit, and the talent were generally amateurs, friends and aspiring actors willing to work well below scale. For this reason I decided on an improv approach, letting the actors basically play themselves, and created the specific branching elements on the set while shooting.

This whole guerilla theatre approach I think lends a lot of credibility to the performances, and the haphazard shooting has a documentary kind of realism.

GS: Sort of the proof that when you want to do something, the thing is to do it.

JG: An obvious and major limitation has to do with where we are situated geographically. Ottawa is hardly a major production centre, and so we knew our talent base would be pretty slim (which problem was solved by the budgetary solution referred to above).

We were also worried that we wouldn't have enough world-class locations to situate it in the big city ambience we were hoping for... On this count I think we did pretty well -- I have been told with some authority by people that they recognized some of the locations... in Paris, London, and New York! Ottawa also has some very pronounced seasons, so you've got to be careful you don't start shooting externals in August and have to do pickups in December!

GS: Your POV is first person in "Midnight Stranger." Was there a reason that you conceived of it that way? Are there advantages to building a story for interactives using a single main character?

JG: This is one way that MS and this style of production have more in common with VR than game playing. In true VR there is only one main character and that is the player/user. Everything that inhibits the suspension of disbelief in the fact that this is an actual experience necessarily reduces its impact. Obviously this has to be done POV, since that is how we experience reality.

There are in fact tremendous disadvantages to trying to simulate reality and tell any kind of multi-pathway story with anything less than infinite memory and unlimited budget. The most important consideration is -- whose POV? The ideal would be to give the user the option of selecting their own persona (avatar, if you will), so that they could 'be' whoever they want, and the characters would respond to them appropriately. In the absence of this fantasy the attempt was made, with varying degrees of success, to create a single set of responses that would somehow work for a user of any race, gender or orientation.

This is also why I often refer to what we do as science fiction -- in form, and not necessarily content. It is a simulation of a future technology within the limitations of the present.

GS: Interactives need to have an additional element that supports the process of interactivity. What did you use in "Midnight Stranger" and in "Mode"?

JG: If I understand this question correctly, then I think it refers in our case to the 'horizontal' navigation elements that allow the player to select where to spend their 'time' in the game environment, which aspect attempts to be an extension of the reality simulation paradigm that informs the entire product.

I very early on decided that I wanted to have almost no traditional gaming devices, that there would be no secret files or onscreen readouts or status bars. Also, and most importantly, there would be no 'winning.' In the real world the decisions you make from moment to moment may improve your life or not, but whatever happens it still is your personal experience, and your 'reward' for all previous choices made. You may guess that things could have been better if you had done things differently, or that you've achieved the best possible experience, but you can't really ever know.

In this simulation the pathways can be explored repeatedly to see what other possible outcomes are available, and you might discover that the true 'rewards' (i.e., entertainment value) might as easily be found on what appears to be a negative as a positive pathway.

Navigation in the physical world of the city nightscape is meant to reflect the kind of realtime potential that occurs for any visitor out on the town for the night. For the most part you can freely wander in and out of public places and you'll tend to meet the same regular patrons or workers, but there is a time factor involved. If you hang out in one place for a long time other places will close, people will leave, the night will end.

We applied this time contraint to what we called our 'endings,' which were devices for encapsulating an experience. If sufficient 'time' has passed in your experience then a character may present you with a concluding statement or action which will serve the function of a final scene in a film, followed by credits. It's obviously unnatural, but then being able to re-boot your night on the town isn't exactly normal either!

As for the specific navigation it is simplicity itself. If you can move in a direction, deeper into a room, say, or to the next street, or into that restaurant, or to approach that person, the cursor changes to GO. When you are close enough to talk to somebody the cursor changes to TALK. There are no other actions except for the few times you find yourself with the mysterious 'object' that is the one fantasy story element built into the title.

GS:. How did the frame-in-frame concept enable you to create your story on the screen?

JG: The frame-in-frame thing was pretty well just a stop-gap measure in my mind, something to create anything like a satisfying experience within the data constraints at hand. I would of course prefer to be working in full-screen video. The choice was really between it and bluescreening the talent and matteing them onto photographic location shots, but I knew from the outset that the 'performance ambience' would be very important, and that my untrained talent would never be able to be natural on a bluescreen set.

And, anyway, the bluescreen technology that we would have been forced to use just looks like crap, as far as I'm concerned, and we would have had all the attendant headaches associated with avoiding key colours and flying hair and all that nuisance. I did tons of bluescreen in my TV show, and it worked okay in the absurdist comedy context, but people recognize the effect almost immediately when they see it and I knew it would be a major blow against that whole suspension-of-disbelief thing if we used it here.

The many times that the registration goes out of whack on the frame-in-frame stuff that is in the final product can be equally distracting, of course, but I feel that the computer mindset associated with windows of information makes it easier to isolate the moving frame as your key information source; the large still is just there to give it graphical ambience, a peripheral context.

As for the story, it is of course a horrible constraint having to have the character with whom you are interacting remain as still as possible throughout a scene! It's a nightmare! Any time that any kind of action was called for we had to jump to a single frame form and use a handheld camera. We called these 'payoff' scenes, since they were what you got at the end of an interaction session as the result of your choices, but were generally themselves non-interactive.

We had to allocate more memory to these scenes as well, since in order to compete graphically with the full-screen images that had come before their moving frame size had to be relatively large. If sufficient time has gone by these payoffs often become endings as well.

Overall the frame-in-frame thing was the necessary evil that allowed there to be enough material on the disc so that there was even the illusion of a story with each of the characters and with the overall product.

GS: What was the process? Was it like (in film language) creating a master shot and building the scene as a derivitive of that?

JG: "Midnight Stranger" and "Mode" had slightly different approaches to the technique. In MS I used a single camera setup, grabbed the wide shot, then zoomed in and locked off for the close up moving frames. The problem with this is that the zoomed image is necessarily substantially different in many respects from the wide. The depth of field changes, backgrounds change, image quality changes. Image quality was a factor, since we were shooting on Hi-8, and wanted the final video to be as good as possible, so we decided to just go with the other problems associated with zooming. With "Mode," since we were shooting on Betacam SP, we decided to try maintaining the wide shot and cropping out the moving frame in post.

GS: It seems like it is an unusual - even tricky - technology. I want to ask, when you shot Mode full screen and cropped the CU's did that help with the dissociative image? I mean was that a better technique than that used in MS?

JG: Yes and no. When I used more common sense and made sure that there was a functional frame edge to work with in the shot, like a table surface, or a way to get the actor to be particularly still, as in one shot with an actor reclining on a big metal chair (it was actually a dissection table), then it worked very well. But whenever I used an actor with a side view, or standing, it caused the most atrocious slippage in the lines that makes them sometimes look like the first special effects frame after the samurai has sliced the head from the shoulders and its starting to slowly slide off.

GS: What is the state of frame-on-frame now?

JG: I wish it would go away! By that I mean I want to work with advanced video compression systems or streaming online technologies or interactive television that allow for full-frame video. We got so sick of the registration problems after the "Mode" experience that for a followup project ("Human Agency") we decided to try a more abstracted approach that abandons registration and embeds the moving image in an impressionistic screen reflective of the overall ambience of the space, and it works quite well.

GS: We'll be looking for it. "Mode" is both a CD and an online game featuring the "mode" bar. What function does this perform for the interactive narrative?

JG: There may be some confusion here. The MOOD BAR is the name I gave to the adaptation of Simon's original emotion input device. From what started as a discrete set of buttons I proposed a seamless colour spectrum band that could have the specific choice elements hidden along its length. This meant that there could be as many or as few triggers as I wanted, and that their transition points could be shifted easily.

On one extreme the character isn't paying any attention to you and it doesn't matter where on the band you click, they'll barrel ahead on their own agenda (making the mood bar no different than a "continue" button). On the other extreme the character could be extremely sensitive to your input and there will be five or six discretely different responses possible, many leading to very different pathways.

We even threw in a few that were random triggers, just for fun! In the main I tried to concentrate on a three-way toggle, but with the transition points shifting to reflect the changing sensitivity of the character. The person might be in a foul mood, and the negative trigger area dominates the bar; they might be in a joyful mood and the positive end will be largest; they might be apathetic and only extremely positive or negative responses will have any effect other than further apathy.

From the perspective of the user that wants to know that they've seen everything on the disc this can be a nuisance, but I believe that by reducing the amount of onscreen information to a minimum, and by constantly adjusting the parameters of the interface device to reflect the changing 'relationship', the illusion of real conversation is strengthened.

The MODE BAR, on the other hand, is the interface device on the game-within-the-game called the Mode Machine in the CD "Mode." To contrast with the Mood Bar this bar has four discrete areas marked with the four letters of the word "MODE". Each of the letters corresponds to a different graphically intense area within the 'game' where one of the characters can be found expounding with a kind of mock solemnity a collection of nonsense phrases, aphorisms, gags, puns, and the like. There is also a "secret" way of reordering the letters on the bar, which can lead you to some secret files that have a bearing on the overall backstory.

GS: What advice can you offer to writers who want to write for new media?

JG: People should only do what they're driven to do, what inspires and excites them, and which they would do for free if they couldn't get paid for it. Doing is everything. It doesn't matter what the venue is, if it has aspects within it that fulfill the first conditions then it's valid. If it's interactivity that excites you then interact, and experiment with that interactivity all the time. You can do this with a telephone as easily as a computer. You can do it on paper with willing subjects, at a party, by mail. People are people and getting to know their response patterns, reaction times, emotional patterns... these all contribute to your ability to produce convincing and effective simulations in any medium. The same goes for multiple pathway storytelling. With a pen and a 99-cent exercise book you can create a hypertext story that you can try out on any willing subject. Obviously the Internet, a website, and a rudimentary knowledge of HTML will give you a huge (and relatively low-cost) tool for experimenting with branching storytelling.

But the first rule is most important. I do not recommend somebody try to get into new media because some commentator predicts it will be bigger than power boat sales. Those that survive and thrive in this area, as in any, will be those for whom it is as natural as breathing.

GS: Seemingly, you are one of those people. What is your current project?

JG: I am in proposal stage on a half-dozen things, only a couple of them in new media. I love all forms of creative expression, and am especially drawn to real-time installation events. I just finished an installation at a local avant garde arts event called The Edge that pushed the Mood Bar boundaries a bit; that was great fun.

Plans with Animatics are always percolating on several fronts..... I'm also trying to finish a cyberpunk novel I've been toying with for many years!

GS: ... Quite a diverse variety of projects. How do you see the future of interactives?

JG: The key to the future of media is immersion. The trend towards ever-more extravagantly realistic theme park experiences, vast movie budgets exploiting every last technological advance in reality simulation, and the expensive theatres and elaborate largescreen surround-sound home setups for experiencing them, are all responding to the public desire for ever-more immersive entertainment. The specifics as to how this will evolve has been debated in science fiction since the beginning, from Huxley's "Feelies" to the Holodeck, but that it's coming is beyond question. That people are going to want to have hyper-real experiences goes without saying, and I have equal certainty that the ways in which this will occur not only include every way that we can presently imagine them but also a substantial number that we cannot.

Whether it's somebody 'being' Picard 'being' Sherlock Holmes on the Holodeck or some investment banker swimming through a William Gibson-esque cyberscape that has converted stock portfolios into a bunch of roiling lifeforms which he manipulates through biofeedback... it really matters very little, we will immerse, and 'reality' will have a different meaning.

From the creative perspective I have no doubt that most of this interactive material (when it's not actual real-time interaction with other real people, of which there will be plenty, of course, in the strangest and most wonderful ways imaginable) will be computer generated on the fly; the writer's job (if they want) can be reduced to defining parameters that will direct the overall experience.

This is not to say that other media will disappear. In fact most areas of expression will flourish side by side, even as they are now for the most part. For example, I have found myself listening to radio while reading a book with an old movie on the TV while I wait for something to download online, jotting point form notes and diagrams in a spiral-bound notepad, having an occasional conversation (with hand gestures) with somebody else in the room... In those moments I am recapitulating the entire history of human communication. The step to adding a direct-implant component that allows me to be doing these things as an Emperor in T'ang Dynasty China, casually seducing Uma Thurman whenever some background story in which she's a time-displaced astronaut brings her into my throne room, is not that large a one as far as I'm concerned, and much to be desired!

How far into the future do you want to go? Ever read any Roger Zelazny?

GS: No, but I will. As soon as I get time. Thanks for putting us on to the Zelanzy reference, Jeff, and the extraordinary interview. You've shed some light on a few interesting interactive techiques. We'll keep an eye out for your work, even if we have trouble keeping up.

© Gloria Stern Hollywood