SkyNet - Co-Op Gameplay (Theory and potential structure)

[hipolipolopigus]

Hey all :happy:

 

Not being able to program isn't a barrier to helping the project! Thinkers and brainstormers are important, too!

Register at http://skynet.hipolipolopig.us to join us!

[Warning! Long post is long, technical and not really suitable for non-programmers!]

 

This is one of the ideas that I've been throwing back and forth in my head while waiting for the CK; SkyNet, Skyrim Co-Op. I can tell that some of you can already feel the need to flame welling up, assuming that this project would fail like so many before it. The thing is that it's not impossible to create this, it's just damned difficult, lengthy and frustrating. Most people have lives that need living so they can't give what they'd like to heir ideas. This is where I come in :wink:

 

I'll tell you a little about me first, if you wanna skip this go right on down to the [Technical Stuff] section. I'm 19, I've been programming for around 8 years, almost 9, in C# and will gladly disregard most other languages as hogwash (BASIC, Java, Objective-C... Not C++. I respect that :P) I first started tinkering with network mechanics in 2009 for a Comp. Sci. class competition thingy and found myself hooked (That might be used as a pun later, we'll see!). I've spent most of my time since learning plenty about serialization, networking structure and (Most recently) CodeDom (Code that compiles code on-the-fly, for those that aren't familiar). Playing Skyrim for a while, I began to feel that it could do with... Well, other people! Could you imagine getting a group together to take down a dragon? (More on difficulty scaling later...) Or raiding Whiterun/Solitude/Windhelm, army against army while 3 or more dragons torch your foes? It sounds fairly lame written down, but I know that I want it.

 

Now, how the heck do we manage it? (Yes, we. Reading this means that you're in on it, too!)

 

[Technical Stuff]

Thanks to the team that have previously worked on OBSE and now SKSE (Pre-release version available here), we have a base for injecting our own code into Skyrim. Until the CK is released, we can't hook anything important (Properly, at least) like players moving, equipping/removing items, entering/leaving combat, etc. But we can at least get started. So, the first thing to do (For me, at least. Being Mr. C# :P) is to inject the .net framework into Skyrim.

 

"Well nuts," I thought to myself when this came up, "That can't be done!" BZZT It can. The ICLRRuntimeHost interface allows us to call up the .net framework inside of a non-managed application when injected (Effectively most modern games). Several issues may crop up with this, however. How does this affect the stability and performance load of the game? Things like this will need to be tested. But it can be done (SKSE will be the cornerstone of this, it handles most of the difficult stuff beautifully).

 

Next on the list is hooking every single little thing that happens on both client and server ends. Hell in a handbasket. This, aside from modifying core-content, will be the most time-consuming. We're not running an amateur show and broadcasting only move/equip/combat events. A Server <-> Client model can't work properly like that. Every time a player goes to loot a corpse, that request will need to be sent to the server, then a response sent to the client to say "This is what's on this corpse, take your pick!", player chooses items (Send message to server; "Took A, B, E") ... You get the picture. Lots of messy code with the potential to go painfully wrong. There are many more questions here; "What about events like talking to an NPC?", "What about quest-line events?", "How are you going to handle mods?" and so forth. All in good time. That mostly fits into modding core content and logic, rather than the server side of things. Let's talk about that now.

 

Modding core logic and content. Definitely the hardest part. Will probably need its' own dedicated team. Skyrim will need to be told to either act as a client or a client/server, each acting differently. The client logic will need to be told to wait to receive replies and events from the server as well as send requests to the server for every final result of each action, whereas the client/server will need to be told to listen for requests from clients, distribute responses/events and handle its' own client. We may as well be re-writing everything from scratch.

 

If you've read all of this, understood it and don't feel the need for flaming, you're the kind of person that would be an invaluable asset to a team if this were to get underway. Well, more underway than it already is. There's already a small-ish code base (Approx 650 lines of working code, excluding comments and not counting single parentheses on single lines) ready to be tested once we're able to hook events. In any case, please contact me if you're interested and have a steady background in C++, C# or modding Bethesda's various series.

 

This will be done, I'm not giving up on this and may the 9 have bloody mercy on me if this falls over. So... Heh, start the flames! :ohdear:

 

Network infrastructure

 

Client-Side loops (Each on its' own thread);

• Listening for packets from Server and filtering them to the appropriate destination (ServerResponse and ServerNotify)
  
• Sending queued ClientRequest packets
  
• Watching Skyrim for events that will queue to send packets (ClientRequest and ClientNotify)
  
• Possible bug-watch thread
  

 

Client/Server loops (Each on its' own thread);

• Single thread for checking each client stream for ClientRequest and ClientNotify packets (As simple as if(!NetworkStream.DataAvailable)continue; thanks to C# ♥) then filtering them to the appropriate target.
  
• Checking for connection attempts and queuing them for verification
  
• Dequeuing and processing connection attempt verifications
  
• Sending ServerNotify packets when receiving a ClientNotify packet
  
• Sending ServerResponse packets in response to ClientRequest packets
  
• Watching Skyrim for events that will queue to send packets (ServerNotify)
  

 

ClientRequest packets are the client requesting something like taking an item from a corpse/chest/other storage unit

ClientNotify packets are the client telling the server that something like the player equipping an item has happened and that the other clients need to be updated

ServerResponse packets are responses to ClientRequest packets containing a trinary value (Yes, no or Detail). "Detail" will provide additional information for clients to operate on.

ServerNotify packets are the server passing on received ClientNotify data to each client that didn't send the received data.

[ClientRequest Structure]

• Header (Byte, 0xA)
  
• RequestType (Byte, will need to enumerate all possible events)
  
• RequestDataLength (Int32)
  
• RequestData (Byte Array)
  

 

[ClientNotify Structure]

• Header (Byte, 0xB)
  
• NotifyType (Byte, will need to enumerate all possible events)
  
• NotifyDataLength (Int32)
  
• NotifyData (Byte Array)
  

 

[ServerNotify Structure]

• Header (Byte, 0xB (May as well recycle the 0xB))
  
• NotifyType (Byte, will need to enumerate all possible events)
  
• NotifyDataLength (Int32)
  
• NotifyData (Byte Array)
  

 

[ServerResponse Structure]

• Header (Byte, 0xC)
  
• ResponseType (Byte, 0x1/0x2/0x3)
  
• == Only included if ResponseType is 0x3 ==
  
• ResponseDataLength (Int32)
  
• ResponseData (Byte Array)
  

 

 

 

The forums are up! :dance: I just sub-domained the URL I got for my filehost :P http://skynet.hipolipolopig.us. Recruitment is a-go!

Edited December 2, 2011 by hipolipolopigus

[Psientist]

Can you please convert the game to Cryengine 3 also?

[hipolipolopigus]

Can you please convert the game to Cryengine 3 also?

Not sure if trolling or...

 

No, and I don't think I'd want to. It's not implementing a new game engine, it's heavily tweaking the existing one to make it bend to our will :happy: Don't mistake the possibility, it can be done (Not the CE3 replacement, retrofitting the existing engine).

Edited December 1, 2011 by hipolipolopigus

[FireBug3]

Injecting framework into skyrim ? thats one bold idea, i wish you luck on that, its not going to be easy.

 

btw i think that guy is serious, you kind of forgetting sometimes that not everybody knows how games are made.

Edited December 1, 2011 by FireBug3

[hipolipolopigus]

Injecting framework into skyrim ? thats one bold idea, i wish you luck on that, its not going to be easy.

 

btw i think that guy is serious, you kind of forgetting sometimes that not everybody knows how games are made.

It can be done, it's working right now (See the part of the first post about ICLRRuntimeHost), we just don't have access to any of Skyrim's functions with it because we don't know what they are. Once we do know them, we'll need to test the impact on Skyrim to see if the project is feasible or not (It should be, won't add much of an FPS hit).

 

Mhm, perhaps... Although sometimes you do need to wonder, troll wannabes are everywhere these days.

Edited December 1, 2011 by hipolipolopigus

[CampanaAliquanta]

How would you handle mods? Wouldn't it have to use vanilla skyrim?

[Szimitar]

Well as most people I doubt this will work, but let's just have some fun and see how this would work if it's possible.

 

First thing we probably have to acknowledge is the fact that this will never be an MMO. Not only is the amount of Data too high to be transfered over average internet connections, the whole design of TES games just doesn't allow for MMO action. Even if it was possible, it just wouldn't be fun. So we should instead focus on local area networks. If someone has Ultra-fast internet and wants to play over Internet they can use Hamachi or something similar, but for most people this is irrelevant. If multiplayer is possible, I'm fairly sure 95% of the users would play it with a small group of 2-10 people over LAN.

 

The next thing to consider is how tasks will be split up over machines. I think the need for a dedicated world server is indisputable. However, cities and other instances could easily be hosted by a different computer and that I think is something that should be used, especially when there are more than 4 or 5 players. Say two or three people enter a cave, the entire instance could be hosted on the first player's computer. Same goes for cities, which can be hosted on a player's PC or a different dedicated server. While this solves a lot of performance issues, it could of course cause a lot of trouble when one of the hosts is disconnected from the network, in which case information would be lost. So we need to figure out a system to temporary allocate players to a seperate server, but to resync information when they leave the instance.

I also think characters should be stored on the client's side. While this does allow for cheating, the target group of this Mod is likely to be a group of friends, so this doesn't matter to much. On top of that, we probably wouldn't have the ressources to create an anti-cheat system anyway.

 

How would you handle mods? Wouldn't it have to use vanilla skyrim?

 

I don't think a multiplayer mod would break any interface or graphic augmentations. Content-mods should also be possible if they are installed on both server and client. Only mods that change gameplay will not work.

Edited December 1, 2011 by Szimitar

[hipolipolopigus]

How would you handle mods? Wouldn't it have to use vanilla skyrim?

Possibly. Although it wouldn't be too difficult to do a hash check of the mod information and compare client <-> server. Or, seeing as the server handles everything, only the server's mods would be utilized. Things like that are still in the drafting stage, just waiting on the CK so SKSE can hook the good stuff for us :happy: Once that's done, work can begin on further researching things like client/server content differences.

[hipolipolopigus]

Well as most people I doubt this will work, but let's just have some fun and see how this would work if it's possible.

 

First thing we probably have to acknowledge is the fact that this will never be an MMO. Not only is the amount of Data too high to be transfered over average internet connections, the whole design of TES games just doesn't allow for MMO action. Even if it was possible, it just wouldn't be fun. So we should instead focus on local area networks. If someone has Ultra-fast internet and wants to play over Internet they can use Hamachi or something similar, but for most people this is irrelevant. If multiplayer is possible, I'm fairly sure 95% of the users would play it with a small group of 2-10 people over LAN.

 

The next thing to consider is how tasks will be split up over machines. I think the need for a dedicated world server is indisputable. However, cities and other instances could easily be hosted by a different computer and that I think is something that should be used, especially when there are more than 4 or 5 players. Say two or three people enter a cave, the entire instance could be hosted on the first player's computer. Same goes for cities, which can be hosted on a player's PC or a different dedicated server. While this solves a lot of performance issues, it could of course cause a lot of trouble when one of the hosts is disconnected from the network, in which case information would be lost. So we need to figure out a system to temporary allocate players to a seperate server, but to resync information when they leave the instance.

I also think characters should be stored on the client's side. While this does allow for cheating, the target group of this Mod is likely to be a group of friends, so this doesn't matter to much. On top of that, we probably wouldn't have the ressources to create an anti-cheat system anyway.

LANs are a most likely going to be a major base, sure, but I don't want to completely exclude remote users. Since the .net framework is involved, it's a matter of pointing a TcpClient to an IP address and port with a TcpListener on it. Nothing more, nothing less. It's not an MMO (Hence why I phrased it Co-Op :happy: ), and the amount of data that can be transferred solely depends on hardware. Admittedly, this can't be run on a dedicated machine in the style of real MMOs like WoW, but the data is minimal if it's designed properly.

 

I've considered the split-LAN model. Having had my finger in several private-server pies for WoW and AION, I can say that this is the model they've implemented (Although due to the low traffic, it's not strictly necessary for them). Although under certain circumstances, systems may be better off with running everything on a solo machine. We'll need to benchmark. Lots.

 

An anti-cheat system wouldn't be out of the question, given the nature of SKSE (Or at least OBSE). As with anything, it's a matter of working it out.

 

Thanks for taking the time for such an exhaustive input! I like that you're using "we" as well :happy: Welcome to the team :thumbsup:

Edited December 1, 2011 by hipolipolopigus

[Jacen000Solo]

I assume that LAN aspect means new saving system. What if you use NWN system (char saved on players pc, savegame (quest progress and so on) on the hosts pc). Instant system sounds reasonable (first to enter runs the instant as a server (there must be a server transfer system so if a current server for the world switches to inst , world server is transfered to another pc). Critical info sent to the host for save. On host disconect (host is the only player online no changes are saved).

 

edit: what i ment with server transfer is a system like windows server clusters.

 

Was it bad?

Edited December 1, 2011 by Jacen000Solo