Simulation Control Center incl. Voice Control for ACC Pitstop MFD

Hi Guys,

I am a great fan of ACC and a passionate Sim and Real World Driver. As many of you as well, I've setup a full motion simulation rig last year during spring to keep my passion going in the Covid crisis. What I've learned, that quite a lot of different software and hardware needs to be integrated and the resulting user experience was not as I expected, since I want maximum immersion. In a car (at least in a race car), I don't want to operate a computer, but I want to control everything by switching switches and push buttons.

The last few months, I have been working on a very sophisticated, extendable and fully configurable Control Software for all aspects of a complex Sim Rig, incl. Motion & Tactile Feedback Systems, but also as an integration center for all the software, we use in our complex simulation setups, including the simulation games itselves, like ACC. The basic idea is, that really everything from starting and stopping a simulation, changing the intensity of a motion effect, and so on, must be controllable by a hardware controller like a button box. but most of the stuff can be controlled by voice as well.



And, since every Sim Rig is different, everything can configured by the user using a graphical configuration tool.



Today, I have released the latest version of the software, which also brings complete control for all pitstop settings of ACC using either a button box or voice recognition as well (as our driving coach always told us - keep the hands on the wheel).

The software incl. the latest release is available on GitHub: https://github.com/SeriousOldMan/Simulator-Controller#simulator-controller and is accompanied by a complete (> 100 page) online documentation as well: https://github.com/SeriousOldMan/Simulator-Controller/wiki

Have fun, TheBigO

 

Very nice tool you've build there.
What draws my attention was

So with only my T300 GT Wheel as button-input controller I'm able to define layers and e.g. for the PS-Buttons (X, square, triangle and circle) define actions for each mode/layer?
That's very neat.

 

Yes, exactly that is the idea....

If you want to dig deeper, take a look at the documentation (https://github.com/SeriousOldMan/Simulator-Controller/wiki/Plugins-&-Modes), to get an overview for all the modes, that are already supported by the standard distribution. But if you have a little programming experience, you can even define your own modes using a simple to use plugin interface.

 

Next version is online, which now includes also driver selection for team races using voice control. Beside that, the hybrid rule engine has been developed and included under the hood. This will be used in the next major release to implement an AI-based Race Engineer which can decide on pitstop strategy, tire pressures and compound, and so on based on real race data.

 

I am currently in the final phase for the next release. Beta testing will begin next week. Greatest news: A fully voice dialog capable virtual race engineer based on AI algorithms. Here is a sneak preview for a typical session with "Jona":

-------------------------------------------------------------------------------------------------------------------------

Jona: "Hi, here is Jona, your race engineer today. You can call me anytime if you have questions. Good luck."

(You hear this or a similar phrase, whenever Jona is ready to interact with you. Typically this is at the beginning of the second lap in a race. From now on Jona might call you, when important information are available, or you can call her/him anytime using one of the key phrases - see the section about [phrase grammars](https://github.com/SeriousOldMan/Simulator-Controller/wiki/Virtual-Race-Engineer#phrase-grammars) on that.)

Driver: "Tell me the tyre temperatures."

Jona: "We have a blue screen here. Windows XP is crap. Give me a minute."

(This answer, or a similar one, means, that Jona does not have enough data at the moment to answer your question. Typically, Jona needs to laps at the beginning of the race or after a pitstop, to be completely up and running)

(A lap later...)

Driver: "Tell me the tyre temperatures."

Jona: "We have the following temperatures: Front left 87. 85 Degrees in the front right tyre. Rear left 93. 91 rear right."

(You can ask for other information as well. See the section about [phrase grammars](https://github.com/SeriousOldMan/Simulator-Controller/wiki/Virtual-Race-Engineer#phrase-grammars) for more information.)

(A few laps later...)

Jona: "Warning. Only 3 laps before you will run out of fuel. Should I update the pitstop strategy now?"

(With this question, Jona asks, if you want to start the preparation process for the upcoming pitstop.)

Driver: "No thank you."

(We are brave and will take another lap.)

Jona: "Okay. Call me when you are ready."

(As you can see, Jona always acjknowledges what you said. This helps you to check, whether Jona did understood you correctly.)

(A lap later...)

Jona: "What are you doing? You are running out of fuel. We should prepare for a pitstop, Okay?"

Driver: "Yes go on."

Jona: "Okay, give me a second."

(A few moments later...)

Jona: "Jona here. I recommend this for pitstop number one: ..."

(Jona gives you a complete overview over all the settings, that were derived for the upcoming pitstop, refueling, tyre compound, pressures, repairs, and so on.)

(Maybe you have some corrections...)

Driver: "Can we decrease front right by zero point two?"

Jona: "The pressure of the front right tyre shall be decreased by 0.2 PSI, is that correct?"

Driver: "Yes please"

(A moment later...)

Jona: "I updated the pitstop plan like you said. Anything else?"

Driver: "No thank you"

Jona: "Understood. I am here."

(The pitstop is now planned, but still not locked in. We are brave and stay out for the last lap...)

Jona: "Warning. You will run out of fuel in one lap. You should come in immediately. Shall I instruct the pit crew?"

(With the last question, Jona aks, whether the pitstop plan shall be locked in.)

Driver: "Ok, let's go on."

Jona: "Okay, I will let the crew prepare everything immediately."

(A few moments later...)

Jona: "We are ready for the pitstop. You can come in."

(The Pitstop MFD window of *Assetto Corsa Competizione* comes to life and all the data is input automatically. See the section about the [pitstop handling](https://github.com/SeriousOldMan/Simulator-Controller/wiki/Virtual-Race-Engineer#the-pitstop) for more information.)

(You enter the pit and bring the car to a stop.)

Jona: "Okay, let the crew do their job. Check ignition, relax and prepare for engine restart."

(And now you are ready for your next stint...)

-------------------------------------------------------------------------------------------------------------------------

The immersion factor is like ten times ten. Stay tuned...

The BigO

 

Thanks for the effort and the voice recognition for MFD is really a must have for endurance races.

So my advice : keep it straightforward with short interaction with the engineer. And please implement the option to call our budy for driver swap!

 

That looks and sounds amazing, top work sir!!

 

A very nice usefull ACC Addon to come ; can't wait to try that !

 

Final release is online. Please be aware, that Jona is still in Alpha stage. So be careful, if you use it during an important race. There might be Gremlins out there...

If you have questions, bugs or even ideas for improvements, feel free to contact me. If the community grows, I will create a Discord server for this, but for now, we can use PMs.

 

Hi TheBigO, just tried your app. Lot of potential with Jona. I'm having an issue each time Jona tries to change pressure, no problem with adding fuel (perhaps it could be could not to refuel above the tank).
It became a loop : going done in the pit MFD enable "change tyre" then goes up , down again disable "change tyre", goes up.... and it also changes the dashboard of the car.
I believe I follow the instructions but...

 

Did you follow the instructions for the picture elements of the Pitstop MFD? See https://github.com/SeriousOldMan/Simulator-Controller/wiki/Update-Notes#release-20

Since ACC has no API for the Pitstop MFD, I use image recognition to understand, in what state the Pitstop MFD is. Quite complicated stuff and the problem is, that the small image elements the program searches on the screen are dependent on screen resolution and language settings.

 

Yes I've done it except for "stint driver 3" that I cannot find in game
Should I try to increase the size of the pictures ? what's the limit ?

 

Hmm, not so easy to debug remotely. What you can do:

1. Start "Simulator Configuration.exe" while holding the control key down.
2. Enable Debug in the lower option group on the General tab and also set Trace Level to "Info".
3. Head on to the track. For a shortcut, you can use the Pitstop mode on the Button Box to control the Pitstop MFD (no physical box needed, you can use the mouse).
4. After that, take a look into the log file "Simulator Controller Logs", which is located in the directory "Simulator Controller\Logs" in your user documents folder. You will find information there about the screen scanning and what the system "sees".

Then we might have a clue, what goes wrong there. What type of screen setup do you use?

 

I tried, but I'm already stuck : when I launch the game the visual button box on the screen hide itself and I cannot see it even when ACC is in windows mode. It's visible again when I shut downACC.
(by the way shift key for me when starting "simulator Configuration.exe")

I'm using single screen 2560x1440

 

I managed to do it with the shortcuts. Here what I get in the log :

I believe it cannot find the selected driver but that normal cause there is no swapping in this race.

 

Well I retoke all the pictures and now it's working !
I'm glad to enjoy it fully

 

Ok, first things first:

1. You can configure whether and how long the button box window stays open using the "Simulator Settings" application. Every value larger than 9999 means practically forever.
2. You are completely right with the shift key. sorry for that...

Now for the log. It looks like it cannot find the "Change" tyre and "Change" brake options (last lines in the log file). Give me a day or two, I will create a test setup with your resolution, maybe you found a bug.

 

Oh, sorry, I haven't seen this post. Great, great. Have fun. And feel free, if there are any other questions...

 

Maintenance release 2.1.3 is online. Release Notes: https://github.com/SeriousOldMan/Simulator-Controller/wiki/Release-Notes#213-release-021021

 

Teaser for the next version: