Greeter screen - a worthwhile MainLobby project?

Tell others how you're using MainLobby.
keyvan
Intermediate
Intermediate
Posts: 373
Joined: Tue Feb 04, 2003 12:35 pm
Location: London - UK
Contact:

Greeter screen - a worthwhile MainLobby project?

Postby keyvan » Fri Jul 07, 2006 2:09 pm

While this is not something I have used ML for, it looks like a cool idea.
When looking at the Home of the year gold award winner:

http://www.electronichouse.com/homeshow ... w/372.html

The house uses 8 ProGears..
I saw this particular application which I thought may be somewhat recreated with Cinemar products:

http://www.fasttrackteam.com/greeterdemo.aspx

They have an avatar to talk to the user but other than that everything else (and more) should be achievable with ML and HomeSeer.

I just wanted to know if anyone has thought of something like this and if yes would they kindly share their ideas / methods?

In the gold winner home, there is a plasma in the entrance hall with the greeter and it looks great!

Keyvan

Keyvan

HawkBlade
Newbie
Newbie
Posts: 20
Joined: Fri Feb 17, 2006 12:39 pm
Contact:

Postby HawkBlade » Mon Jul 10, 2006 11:26 am

AHH, to have lots of money to have the "new" toy's....

synchroman
Intermediate
Intermediate
Posts: 388
Joined: Fri Jun 10, 2005 12:39 am
Location: Los Angeles, CA
Contact:

Postby synchroman » Mon Jul 10, 2006 2:38 pm


Teece
New Member
New Member
Posts: 43
Joined: Wed Jun 29, 2005 8:44 am
Contact:

Postby Teece » Wed Jul 26, 2006 10:20 am

Has anyone gotten a talking avatar to work with Main Lobby? I have HAL2000 and would love to get an avatar working with my system. Kind of a crude method, but couldn't you get a .swf file of a talking head and trigger it to play when a TTS file is read? Does anyone have any ideas on how to get something like this to work?

cpetro
Newbie
Newbie
Posts: 15
Joined: Mon Mar 20, 2006 12:44 am
Contact:

Avatar

Postby cpetro » Wed Jul 26, 2006 3:57 pm

Teece,

Sorry this is so long, but here we go.
I use HAL Deluxe and saw the avatar post on the HAL forum. Got me thinking how to do this. My thoughts were pretty much the same as you stated above. This works, but only to a certain degree. You need some Photoshop/Flash skills to do this. First get/draw a picture of the avatar head you want to use. Use Photoshop to edit the images that are going to be animated. You will want to create a couple of different graphics for each level of eye/mouth motion. Import this into Flash and lay out your animation. In my case the head is always stationary, the eyes blink at a few different intervals , the lips move slightly when doing nothing and open and close when talking. The animation works awesome, it looks realistic like a video game character, but as stated above combining with HAL isn't flawlwss.

Now for the scripting. In HAL make two rules: HAL is about to speak, and Hal has finished speaking. Use the command line in HAL to send ML Server a variable change for each state.

Here are the problems with HAL.

1. There is a delay between the sync of the animation and the TTS. If the TTS is short, by the time the variable changes and the animation starts the TTS might be finished, causing the variable to change back. This causes sync issues or missed triggers.

2. No way to determine how speech patterns correlate to lip movement. This would have to be hardcoded in HAL to recognize from TTS. Dont even know if it is possible. Only different levels of general lip movement are available.

3. This only works if the TTS is triggered by a schedule or an If/Then rule. My inital thought in doing this was to speak to HAL through the open air mics upon arrival at home and have a visual on the screen. You cannot have this trigger when speaking to HAL through the microphones. The HAL is about to speak rule doesn't work because it doesn't consider HAL is listening through microphone as doing TTS, so no variables are changed. Even if it did the mouth would still be moving when the avatar was not speaking and waiting for a reply from you, this makes speaking to the avatar pretty unrealistic. HAL would need to expanded on the command structure to start and stop the mouth appropriately. So as of right now only triggered speech with some lag.

As far as the ML side of it, there are two ways to do this.

1. Have two swf files, one stationary and one talking. Use ML Server variables to dynamically change them.

2. Embed one swf with the ML variables and conditional logic. This gets a little more complicated. Script the first frame to determine the ML variable state. Frames 2 thru 9 are just eye blinks. Frames 10 thru 17 are eye blinks and lip movement. If not talking goto frame 2....If talking goto frame 10. On frames 9 and 17 put actionscript goto 1.

Example:

Frame 1 loads determines variable state, goes to appropriate frames and plays, gets to end(frame 9 or 17) and goes back to 1, if variable is the same as it was it loops until it changes, when variable changes frame 1 sends animation to new set of frames.

This eliminates the need for using reload every second which only lets you see the first second of the animation repeating if it is longer than 1 second, which it probably will be. The problem with the example above is if the animation is long and a variable changes half way through, it has to wait until the end of the frame go back to 1 to check the variable state.
To fix this for long animations have actionscript applied to every frame not just the end, so if the variable changes in the middle of an animation it evaluates the state on that frame and goes to the correct frame loop for the new variable. This eliminates alot of lag.

If you want you can add variables and frames for different emotions, eyebrows up/down, eyes wide/closed, lip movement allows for many expressions. Very cool. Weird thing though, sometimes when the scene starts the avatar is frozen(looks like shes is dead, eyes half open)I have to go into edit and back to launch to fix her. Probably needs some of Mario's secret code to display correctly since it has conditional actionscript in it.

That is what I have figured out from experimenting with this. This could be very cool if the problems could be worked out. Anyone with ideas please post.

Chris

User avatar
jeffx
Intermediate
Intermediate
Posts: 256
Joined: Mon Jun 26, 2006 9:36 am
Location: Upstate NY
Contact:

Postby jeffx » Thu Jul 27, 2006 7:50 am

take a look at People Putty:

http://www.haptek.com/
HAL2000 - JRMC 11 - ML3 - MuL3 - NetRemote Pro 1 - Insteon - M-Audio Delta 410 - Sonic Impact T-Amps - Honeywell Security

HawkBlade
Newbie
Newbie
Posts: 20
Joined: Fri Feb 17, 2006 12:39 pm
Contact:

Postby HawkBlade » Thu Jul 27, 2006 11:10 am

Hey mario.... maybe a nice thought for ML4???

Adding in a avatar, or support for them.

User avatar
jeffx
Intermediate
Intermediate
Posts: 256
Joined: Mon Jun 26, 2006 9:36 am
Location: Upstate NY
Contact:

Postby jeffx » Thu Jul 27, 2006 12:05 pm

or even ML 3.2 :D
HAL2000 - JRMC 11 - ML3 - MuL3 - NetRemote Pro 1 - Insteon - M-Audio Delta 410 - Sonic Impact T-Amps - Honeywell Security

User avatar
mcascio
Founder
Founder
Posts: 8270
Joined: Tue Jan 28, 2003 4:03 pm
Location: Kenosha, Wi
Contact:

Postby mcascio » Thu Jul 27, 2006 12:17 pm

My theory is that you should be able to use their program ala WebLobby today.

If someone wants to play more with their program and report back on how to get text>speech - you could be in business sooner than you think.
Mario Cascio
Cinemar, Inc.
http://www.cinemaronline.com

Teece
New Member
New Member
Posts: 43
Joined: Wed Jun 29, 2005 8:44 am
Contact:

Postby Teece » Tue Aug 01, 2006 2:10 pm

Thanks Chris! I had already created two .swf files of my avatar. One speaking and one sitting pretty, I haven't gotten around to playing any further, I've been on vacation. Can you post your HAL/ML logic for the variables? It's been so long since I did any of this, but since I've moved into a new house and gotten a new computer I now have a lot of updating to do. Once I get the two flash files working with HAL, I'll look into creating one file with a large number of frames. Thanks again for the response!

As for Putty People I'd try it, it looks like a great program, but I'm having a hard time seeing how I could get it to work via WebLobby and HAL.

cpetro
Newbie
Newbie
Posts: 15
Joined: Mon Mar 20, 2006 12:44 am
Contact:

Avatar

Postby cpetro » Thu Aug 03, 2006 7:27 pm

Teece,

The method using two swf files is as follows:

Create ML button
Put {{avatar}} on Text line (ML2.86b)
Put {{avatar}} on JPG/SWF line (3.0)


Rule 1: HAL IS ABOUT TO SPEAK
IF Hal is about to speak through speakers
THEN run program MLServer.exe

MLServeCmd.SetVariable|avatar~C:/Program Files/Cinemar/images/avatar talking.swf


Rule 2: HAL HAS FINISHED SPEAKING
IF Hal is finished speaking through speakers
THEN run program MLServer.exe

MLServeCmd.SetVariable|avatar~C:/Program Files/Cinemar/images/avatar waiting.swf

HAL can fire any ML command by using Run Program, select MLServer.exe from the directory and put the MLServe command in the parameters line.

I messed around with this some more and it barely even works. I thought the fact that the TTS was short was causing the delay issues, so I made a really long TTS. Even though it is suppose to do the HAL is about to speak command before it fires the TTS, looking at the log times in ML and HAL it always changes the variable after the TTS. Don't know why. So really the longer the TTS message is the worse it works. If the message was three lines of typed text and was executed at 1:00:00, rule #1 would fire at 1:00:15 and rule #2 at 1:00:15. So it was 15 seconds until the talking swf started and only ran for less than a second. Barely saw it. I'll mess with this a little more but will probably abandon this project if no results are achieved. If I figure anything out I will post.

I spoke to Greg from HAL and he said that they had talked to the company that makes the avatar from the Electronic House article. Don't know if they are going to add this to their software, this but at least we know it's on their minds.

Good Luck,
Chris

Teece
New Member
New Member
Posts: 43
Joined: Wed Jun 29, 2005 8:44 am
Contact:

Postby Teece » Fri Aug 04, 2006 7:46 am

Thanks Chris, your info was very much appreciated! I'll see what I can come up with this weekend when I have more time to play around. Thanks again.


Return to “How are you using MainLobby?”

Who is online

Users browsing this forum: No registered users and 1 guest