[LE] SKSE plugin creation, C++, etc.

[irswat]

So I'm working on this voice command engine for skyrim. It was brought to my attention that there was a better way to develop this involving the creation of an SKSE library that calls functions from windows voice recognition API, and returns a string of the recognized dictation to papyrus. In the library I need two things:

1.) A function (lets call it GetSpeechText) that interfaces with SAPI, gets a text string from windows speech recognition API. and returns that string to Papyrus.

Here is an example of SAPI interfacing code from MSDN:

 

#include "sapi.h"

// Declare local identifiers:
HRESULT                   hr = S_OK;
CComPtr<ISpRecoResult>    cpRecoResult;
SPPHRASE                  *pPhrase;
WCHAR                     *pwszText;

// ... Obtain a recognition result object from the recognizer ...

// Get the recognized phrase object.
hr = cpRecoResult->GetPhrase(&pPhrase;);

if (SUCCEEDED (hr))
{
   // Get the phrase's entire text string, excluding replacements.
   hr = cpRecoResult->GetText(SP_GETWHOLEPHRASE, SP_GETWHOLEPHRASE, FALSE, &pwszText;, NULL);
}

if (SUCCEEDED (hr))
{
   return ISpRecoResult
}

 

2.) The library also needs to register with SKSE an EventHandler (OnKeyRelease) like this one, in order to pass the string obtained from SAPI to the parser script in papyrus.

From what I understand I could then call the function in the SKSE .dll from the papyrus parsing engine script. Something like:

event OnKeyPress()
VoiceCMDStr=MySKSEPlugin.GetSpeechText()

endEvent


Now I've never made a SKSE plugin and I haven't done any coding in C++ in years, but this seems straightforward enough that I could do it, especially with all the sample code out there. So I went and downloaded the example plugin over at the SKSE plugin tutorial page. I added the above code to the MyPluginExample Namespace, as instructed in the tutorial, and the compiler wigs out with all sorts of syntax errors.

Is there anyone that would be kind enough to help me figure this out? I can code in C++ but my knowledge is 13 years old, and limited to high school AP C++ stuff. Nevertheless I'm a fast learner, and this one piece of functionality is the last piece of the puzzle to making this voice command engine work.

Edited November 23, 2016 by irswat

[irswat]

COMPILER LOG

 

 

Severity Code Description Project File Line Suppression State

Error (active) identifier "CComPtr" is undefined plugin_example c:\Users\Anonymous\Desktop\plugin_example3\plugin_example\plugin_example\MyPlugin.cpp 8

Error (active) type name is not allowed plugin_example c:\Users\Anonymous\Desktop\plugin_example3\plugin_example\plugin_example\MyPlugin.cpp 8

Error (active) identifier "cpRecoResult" is undefined plugin_example c:\Users\Anonymous\Desktop\plugin_example3\plugin_example\plugin_example\MyPlugin.cpp 8

Error (active) expected a ')' plugin_example c:\Users\Anonymous\Desktop\plugin_example3\plugin_example\plugin_example\MyPlugin.cpp 15

Error (active) expected an expression plugin_example c:\Users\Anonymous\Desktop\plugin_example3\plugin_example\plugin_example\MyPlugin.cpp 15

Error (active) expected a ')' plugin_example c:\Users\Anonymous\Desktop\plugin_example3\plugin_example\plugin_example\MyPlugin.cpp 20

Error (active) expected an expression plugin_example c:\Users\Anonymous\Desktop\plugin_example3\plugin_example\plugin_example\MyPlugin.cpp 20

Error (active) type name is not allowed plugin_example c:\Users\Anonymous\Desktop\plugin_example3\plugin_example\plugin_example\MyPlugin.cpp 25

Error (active) expected a ';' plugin_example c:\Users\Anonymous\Desktop\plugin_example3\plugin_example\plugin_example\MyPlugin.cpp 26

Error (active) incomplete type is not allowed plugin_example c:\Users\Anonymous\Desktop\plugin_example3\plugin_example\skse\PapyrusVM.h 409

Error (active) incomplete type is not allowed plugin_example c:\Users\Anonymous\Desktop\plugin_example3\plugin_example\skse\PapyrusVM.h 446

Error C2065 'CComPtr': undeclared identifier plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 8

Error C2275 'ISpRecoResult': illegal use of this type as an expression plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 8

Error C2065 'cpRecoResult': undeclared identifier plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 8

Error C2065 'cpRecoResult': undeclared identifier plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 15

Error C2227 left of '->GetPhrase' must point to class/struct/union/generic type plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 15

Error C2059 syntax error: ';' plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 15

Error C2059 syntax error: ')' plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 15

Error C2065 'cpRecoResult': undeclared identifier plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 20

Error C2227 left of '->GetText' must point to class/struct/union/generic type plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 20

Error C2059 syntax error: ';' plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 20

Error C2059 syntax error: ',' plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 20

Error C2059 syntax error: ')' plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 20

Error C2143 syntax error: missing ';' before '}' plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 26

Error C2275 'ISpRecoResult': illegal use of this type as an expression plugin_example c:\users\anonymous\desktop\plugin_example3\plugin_example\plugin_example\myplugin.cpp 26

 

code from myplugin.cpp

 

#include "MyPlugin.h"
#include "sapi.h"

namespace MyPluginNamespace {
	float MyTest(StaticFunctionTag *base) {
		// Declare local identifiers:
		HRESULT                   hr = S_OK;
		CComPtr<ISpRecoResult>    cpRecoResult;
		SPPHRASE                  *pPhrase;
		WCHAR                     *pwszText;

		// ... Obtain a recognition result object from the recognizer ...

		// Get the recognized phrase object.
		hr = cpRecoResult->GetPhrase(&pPhrase;);

		if (SUCCEEDED(hr))
		{
			// Get the phrase's entire text string, excluding replacements.
			hr = cpRecoResult->GetText(SP_GETWHOLEPHRASE, SP_GETWHOLEPHRASE, FALSE, &pwszText; , NULL);
		}

		if (SUCCEEDED(hr))
		{
			return ISpRecoResult
		}
	}

	bool RegisterFuncs(VMClassRegistry* registry) {
		registry->RegisterFunction(
			new NativeFunction0 <StaticFunctionTag, float>("MyTest", "MyPluginScript", MyPluginNamespace::MyTest, registry));

		return true;
	}
} 

 

 

[irswat]

 

 

#include "MyPlugin.h"
#include "voce.h"
#include "string.h""
#include <stdio.h>
#include <stdlib.h>

#ifdef WIN32
#include <windows.h>
#else
#include <unistd.h>
#endif


namespace MyPluginNamespace {
	float MyTest(StaticFunctionTag *base) {
		bool quit = false;

		voce::init("../../../lib", false, true, "./grammar", "digits");
		while (!quit)
		{
			// Normally, applications would do application-specific things 
			// here.  For this sample, we'll just sleep for a little bit.
#ifdef WIN32
			::Sleep(200);
#else
			usleep(200);
#endif

			while (voce::getRecognizerQueueSize() > 0)
			{
				std::string s = voce::popRecognizedString();

				// Check if the string contains 'quit'.
				if (std::string::npos != s.rfind("quit"))
				{
					quit = true;
				}

				
			}
		}
		
		voce::destroy();
		return 0;
}

	bool RegisterFuncs(VMClassRegistry* registry) {
		registry->RegisterFunction(
			new NativeFunction0 <StaticFunctionTag, float>("MyTest", "MyPluginScript", MyPluginNamespace::MyTest, registry));

		return true;
	}
} 

 

Please don't laugh. So I thought I would try using a C++ library called voce. It seems to expose the SAPI making it easier to access for entry level coders. So I loaded up the example plugin. I don't really know how to use namespaces because all my work in c++ was using the std namespace. I did some brief reading and learned this is where I would declare functions. I messed around with the above code for some time, first I tried simply returning s, but that obviously didn't work because the function is defined to return a float. The above is the only version I was able to get to compile. Unfortunately it is useless.

I then had what I thought was a eureka moment, and I thought I understood what was going on, and changed the above to the below:

 

 

#include "SKSE_Speech.h"
#include "voce.h"
#include "string.h"
#include <stdio.h>
#include <stdlib.h>

#ifdef WIN32
#include <windows.h>
#else
#include <unistd.h>
#endif


namespace SpeechRecNamespace {
	std::string GetTextFromSpeech(StaticFunctionTag *base) {
		bool quit = false;

		voce::init("../../../lib", false, true, "./grammar", "digits");
		while (!quit)
		{
			// Normally, applications would do application-specific things 
			// here.  For this sample, we'll just sleep for a little bit.
#ifdef WIN32
			::Sleep(200);
#else
			usleep(200);
#endif

			while (voce::getRecognizerQueueSize() > 0)
			{
				std::string s = voce::popRecognizedString();

				// Check if the string contains 'quit'.
				if (std::string::npos != s.rfind("quit"))
				{
					quit = true;
				}

				
			}
		}
		
		voce::destroy();
		return s;
}

	bool RegisterFuncs(VMClassRegistry* registry) {
		registry->RegisterFunction(
			new NativeFunction0 <StaticFunctionTag, string>("GetTextFromSpeech", "SKSE_SpeechRecPlugin", SpeechRecNamespace::GetTextFromSpeech, registry));

		return true;
	}
} 

 

I changed the appropriate function definition names in main.cpp and myplugin.h. I get tons of compiler errors.

I'm used to including the necessary headers, declaring and defining functions and variables in the std namespace, and jumping right into my int main() function. VS 15 is brand new to me.

If there is anyone who can help me understand how to create a plugin using the sample plugin that calls the SAPI and returns a string to papyrus from the speech recognizer I would be very grateful.

Edited November 25, 2016 by irswat

[irswat]

I did manage to build my first plugin. When called from papyrus it returns BSFixedString that says "Hello World!" So now the challenge is how to fill that same string with the text returned from the speech recognition API? For example if I want to use the VOCE library, this seems to use the std namespace. Here is an example:

// Speech recognition in C++
while (voce::getRecognizerQueueSize() > 0)
{
      std::string s = voce::popRecognizedString();
      std::cout << "You said: " << s << std::endl;
}

This piece of code builds, but when I try to call it from papyrus, passing true/false, I get compiler error too many arguments. Clearly I have the function declared to receive one parameter. If anyone could lend insight that'd be cool:

 

#include "SKSE_Speech.h"
#include "voce.h"
#include "string.h"
#include <stdio.h>
#include <stdlib.h>
#include <sstream>
#include <fstream>

namespace SpeechRecNamespace {
	BSFixedString GetTextFromSpeech(StaticFunctionTag *base, bool Listening) {
		const char* c_GetSpeech = "";
		const char* c_TextFromSpeech;

		while (voce::getRecognizerQueueSize() > 0) && (Listening==true)
			{
				std::string s_TextFromSpeech = voce::popRecognizedString();
				const char* c_GetSpeech = s_TextFromSpeech.c_str();
				//strcat(c_TextFromSpeech, c_GetSpeech);
				
				std::fstream myfile("C:\Games\SteamApps\common\Skyrim\Data\SVE_PI.txt", std::fstream::in | std::fstream::out | std::fstream::app);
				if (myfile.is_open())
				{
					myfile << c_GetSpeech << "/n";
					myfile.close();
				}				
			}

		return BSFixedString(c_GetSpeech);
	}

	bool RegisterFuncs(VMClassRegistry* registry) {
		registry->RegisterFunction(
			new NativeFunction1 <StaticFunctionTag, BSFixedString, bool>("GetTextFromSpeech", "SKSE_SpeechRecPlugin", SpeechRecNamespace::GetTextFromSpeech, registry));

		return true;
	}
}

 

 

Edited November 26, 2016 by irswat