irswat Posted November 21, 2016 Share Posted November 21, 2016 I'm working on a voice command engine for skyrim. It is coming along very rapidly, and so far seems to be working better than expected. Right now the engine is able to parse, recognize, and execute weapon equip commands, as well as potion commands. The papyrus engine dumps these commands from a text file.The text file will be generated by windows speech recognition software in conjunction with an as of yet undeveloped program which will simply ensure the text file autosaves, and is formatted correctly for the engines usage. I plan on writing this program in C++. In short to start skyrim with voice command engine you will open this c++ program which will simultaneously initialize the text file, open windows speech recognition, and then launch SKSE. While the program is running the applet will detect when players release the 'mic' button, it will then quickly append literal carriage returns after the voice command delimiter ("QUICK"), start a new line, save the file, wait for the file to be read by the papyrus script, and then reset the output file for the next command. Pretty simple really.I'm searching for one person who is willing and able to help me finish developing this engine, and once release hits, to help work out bugs, and continue support for at least 6 months. I'd be especially interested in someone who has experience with multi threading papyrus scripts. I've been toying with several efficiency and performance methods in the code, but I think multi threading the code will take the engine to the next level as far as responsiveness is concerned. I work, and goto school full time but am committed to this project enough to dedicate at least 20 hours a week to development. If you are a capable scripter and programmer in a language other than Papyrus, and know your way around the CK, then send me a PM. Link to comment Share on other sites More sharing options...
irswat Posted November 21, 2016 Author Share Posted November 21, 2016 Here is an example of one of the time consuming tasks that I could use help with: WeaponNames=new string[10]WeaponNames[1]="Woe"WeaponNames[2]="Nightingale"WeaponNames[3]="Miraak"WeaponNames[4]="Magnus"WeaponNames[5]="of"WeaponNames[6]AegusvabeWeaponNames[7]Alessandra'sWeaponNames[8]AlterationWeaponNames[9]Amren'sWeaponNames[10]AncientWeaponNames[11]Angi'sWeaponNames[12]AnimusWeaponNames[13]AnnihilatingWeaponNames[14]ArcingWeaponNames[15]BalckbowWeaponNames[16]Balgruuf'sWeaponNames[17]BanishingWeaponNames[18]Barvir'sWeaponNames[19]BindingWeaponNames[20]BlackbladeWeaponNames[21]BlazeWeaponNames[22]BlessedWeaponNames[23]BlizzardsWeaponNames[24]BloodthornWeaponNames[25]Bolar'sWeaponNames[26]BoundWeaponNames[27]BriarheartWeaponNames[28]BrokenWeaponNames[29]BurningWeaponNames[30]BurningWeaponNames[31]CeremonialWeaponNames[32]CharmingWeaponNames[33]ChillrendWeaponNames[34]ChillsWeaponNames[35]ColdWeaponNames[36]ConjurationWeaponNames[37]CowardiceWeaponNames[38]DamnationWeaponNames[39]DawnbreakerWeaponNames[40]DebilitationWeaponNames[41]DebilitationWeaponNames[42]DepletingWeaponNames[43]DespairWeaponNames[44]DestructionWeaponNames[45]DevouringWeaponNames[46]DiminishingWeaponNames[47]DismayWeaponNames[48]DragonWeaponNames[49]DragonbaneWeaponNames[50]DrainbloodWeaponNames[51]DrainheartWeaponNames[52]DrainingWeaponNames[53]DrainspellWeaponNames[54]Dravin'sWeaponNames[55]DreadWeaponNames[56]EdujWeaponNames[57]EmbersWeaponNames[58]EndWeaponNames[59]EnervatingWeaponNames[60]EvokingWeaponNames[61]ExhaustionWeaponNames[62]ExpellingWeaponNames[63]EyeWeaponNames[64]FatigueWeaponNames[65]FearWeaponNames[66]Firiniel'sWeaponNames[67]FlamesWeaponNames[68]FreezingWeaponNames[69]Froki'sWeaponNames[70]FrostWeaponNames[71]Gadnor'sWeaponNames[72]GarneringWeaponNames[73]GauldorWeaponNames[74]GeisWeaponNames[75]Ghorbash'sWeaponNames[76]GhostbladeWeaponNames[77]GleaningWeaponNames[78]HarrowingWeaponNames[79]HarvestingWeaponNames[80]HuntWeaponNames[81]HuntingWeaponNames[82]IceWeaponNames[83]ImmobilizingWeaponNames[84]InfernoWeaponNames[85]LeechingWeaponNames[86]LethargyWeaponNames[87]LightningWeaponNames[88]MaledictionWeaponNames[89]MelkaWeaponNames[90]NullifyingWeaponNames[91]OathbladeWeaponNames[92]PetrifyingWeaponNames[93]PriestWeaponNames[94]ReapingWeaponNames[95]ReverentWeaponNames[96]SacrificeWeaponNames[97]SanctifiedWeaponNames[98]SappingWeaponNames[99]ScorchingWeaponNames[100]ShocksWeaponNames[101]SiphoningWeaponNames[102]SorcererWeaponNames[103]Soul SnaresWeaponNames[104]SoulsWeaponNames[105]SparksWeaponNames[106]SphereWeaponNames[107]StormsWeaponNames[108]SubsumingWeaponNames[109]SunningWeaponNames[110]SuppleWeaponNames[111]TerrorWeaponNames[112]theWeaponNames[113]ThunderboltsWeaponNames[114]TorporWeaponNames[115]VampireWeaponNames[116]WearinessWeaponNames[117]WinnowingWeaponNames[118]Winter Link to comment Share on other sites More sharing options...
irswat Posted November 22, 2016 Author Share Posted November 22, 2016 equip weapons + equip hand, equip armor, use potions + numbers of potions, and now shouts, are all working. I'm still looking for help with this. I especially could use help with the C++ helper program, as well as optimizing and multi-threading my code. The requirements of the helper app are simple enough that I could write it. Voice command engine for skyrim? anyone? Link to comment Share on other sites More sharing options...
irswat Posted November 23, 2016 Author Share Posted November 23, 2016 (edited) I ran into a significant obstacle this evening. I hope someone is able to lend some expertise.So I'm writing a text parsing engine for skyrim, and it's coming along very nicely. I'm about 2000 lines deep and it includes support for all vanilla+dlc shouts, a ton of weapons, armor, and potions. It is very easy to add support for new content.The plan was use microsoft voice to speech in the background with notepad to output a text file. A lightweight c++ applet would run along side to format the textfile when the mic button is pressed/released, save the text file, then delete it about a second later. The parsing engine dumps that text file to a string and does all sorts of neat things. The problem is that microsoft voice recognition does not stay focused on notepad when skyrim is open, even in windowed mode. I've considered several options, but as of right now with no one expressing interest in helping me develop this I am at a loss. I'd hate to see this really cool parsing engine go to waste, so the second I can find someone willing to help me develop a solution that makes it possible to dictate speech to text while in game, I'll be ready to get back into development full steam.I've thought about experimenting with the cmu sphinx libraries, but again I go to school full time and work, and I really don't have it in me to try to learn a new programming language.If anyone wants to take a gander at the source code I'm working on, let me know. If you can help leave a message or pm me. Edited November 23, 2016 by irswat Link to comment Share on other sites More sharing options...
ymf331 Posted November 23, 2016 Share Posted November 23, 2016 (edited) if anything and regardless of which speech library you use, trying to dictate into notepad and use that as input for your program doesn't really seem like a robust solution. i'd think that you'd want whatever you're working on to accept voice commands as input just as your basic starting point. you don't need to learn a new language, because there's a library for c++ that has a whole 8 functions that would work fine. http://voce.sourceforge.net/ Edited November 23, 2016 by ymf331 Link to comment Share on other sites More sharing options...
utopium Posted November 23, 2016 Share Posted November 23, 2016 The method you are proposing is highly inefficient and prone to many problems. This is better done as an SKSE plugin instead of an external program without any need to save to text files and such. You can register event handlers so that events from an SKSE plugin can call directly into Papyrus code. Take a look at the SKSE source code for how OnKeyDown is performed for example: https://github.com/xanderdunn/skaar/blob/b20ff15c9bf9a5cdcfecdfd5bf9b70db5e7e56ab/skaar_skse_plugin/skse/PapyrusEvents.cpp#L180 You should be able to use the Windows Speech API to pass the strings produced from the speech-to-text directly to Papyrus using this method in a much simpler and efficient manner and you don't have to worry about complex threading issues as everything is event based. Sorry to say I can't help you with your project, but it will make a big difference if you take some time understanding the SKSE code in order to develop an event based plugin to do what you require. Link to comment Share on other sites More sharing options...
irswat Posted November 23, 2016 Author Share Posted November 23, 2016 Very useful I formation.I knew nothing about the skse plugin capabilities. So it sounds like I can register for a key release in papyrus then when detected call a function compiled into an skse library that returns a string from speech recognition api? Very coolMaybe ill write the parser in c++ compiled it into a skse library and only use papyrus to execute commands. Thanks much guys. Very helpful. Link to comment Share on other sites More sharing options...
irswat Posted November 23, 2016 Author Share Posted November 23, 2016 (edited) I found this, which seems to be exactly what I'm looking for. Maybe I won't even need the voce library Edited November 23, 2016 by irswat Link to comment Share on other sites More sharing options...
irswat Posted November 23, 2016 Author Share Posted November 23, 2016 I'm excited. I did not realize that SKSE was so powerful an interface! Link to comment Share on other sites More sharing options...
irswat Posted November 23, 2016 Author Share Posted November 23, 2016 #include "MyPlugin.h" #include "sapi.h" namespace MyPluginNamespace { // Declare local identifiers: HRESULT hr = S_OK; CComPtr<ISpRecoResult> cpRecoResult; SPPHRASE *pPhrase; WCHAR *pwszText; // ... Obtain a recognition result object from the recognizer ... // Get the recognized phrase object. hr = cpRecoResult->GetPhrase(&pPhrase;); if (SUCCEEDED(hr)) { // Get the phrase's entire text string, including replacements. hr = cpRecoResult->GetText(SP_GETWHOLEPHRASE, SP_GETWHOLEPHRASE, TRUE, &pwszText; , NULL); } if (SUCCEEDED(hr)) { // Get the phrase's first 2 words, excluding replacements. hr = cpRecoResult->GetText(pPhrase->Rule.ulFirstElement, 2, FALSE, &pwszText; , NULL); } if (SUCCEEDED(hr)) { return cpRecoResult; } } bool RegisterFuncs(VMClassRegistry* registry) { registry->RegisterFunction( new NativeFunction0 <StaticFunctionTag, float>("MyTest", "MyPluginScript", MyPluginNamespace::MyTest, registry)); return true; } } I tried making an SKSE plugin using SAPI 5.4 sample code and it just won't compile. It's been like 13 years since I've done anything in C++, and even then it as just high school AP C++ stuff. Link to comment Share on other sites More sharing options...
Recommended Posts