[Moonshot] Deep Neural Nexus

[Runestones]

I first brought this discussion to the Skyrim forums but as it did not have much activity there, I am bringing it to the Fallout forums as I believe there to be more movement here.
As this cannot quite be described as a mod request, I am posting it at the mod talk section, but I'll also be posting it at the NMM Code Development Chat.

 

First of all, I have to ask, Do you know what a "Deep Neural Network" is?
It is the combination of "Deep Learning" algorithm and an "Artificial Neural Network".

-Deep learning: a branch of machine learning that tries to create a multi-purpose algorithm to "learn something" out of a set of data.
-Artificial Neural Network: a biologically inspired programming paradigm that allows a computer to learn by observation and analysis.

So, a "Deep Neural Network" would be the combination of both, a computer program/algorithm that can learn to do something new by observation and analysis, for example, playing a game, learning by itself and getting better by playing, but it can do much, much more than that, as it can learn just about anything that a human could.

Deep Neural Networks have several applications besides teaching a computer how to game. You might have heard, for example, of a website called deepart.io or a cellphone app called Prisma. I can't be sure about the latter, though I do am pretty sure, but the first does use a Deep Neural Network to implement the style of an image or painting into a photo you give it, and it gets better the more it is used, as with each image, it learns how to do it better.


So in theory, a Deep Neural Network could even learn to mod. Even though I would not directly expect for one to just create mods out of our wishes right from the very beginning, it is entirely possible to create a Deep Neural Network that is able to bug-check and fix any problems.

Just imagine a software, able to analyse the code of any mod, understand it, make a game run with it through a cloud gaming server, testing several different mods at once, recombining them, bug checking, bug fixing, correcting incompatibilities and so on.

It downloads each and every game mod in the nexus, steam and any other mods portal, understands it and saves that knowledge for when someone adds a mod to their nexus. This nexus then, instead of installing the mod as it is, would reach the database for every mod the person wishes to add for their game, combine them in a single "super mod" including the DLC's, download this super mod and that is what would be installed.

So it would even remove the limit to how many mods you can have installed as it would actually be a single mod hand tailored to each person individually and should be in a file type that only the program itself can handle, which would also not accept receiving a manual input of a file of that type, only downloading it directly from the cloud.

I believe no one would have a problem with a modified version of the mod they created like that, if it is hand tailored to each person's case.

Imagine it like a "professional modder" that goes to your house, for free, and messes a bit with each mod just to make it perfect for you, but puts it in a way that you can't share it with someone else and you have to call it again to change that mod to add or remove things from and to your game.
You'd have to give your friends the original mods you used and tell them to call that modder to do the same for them. They will have the exact same result, but no one downloaded a modified version of any of the original mods. There is no redistribution of modified content, only fixing and making things compatible. Adding and removing mods and DLC's would be about the same as it already is, the result and amount of work needed is what would be different for it to run smoothly, as it would be way easier and faster...

Maybe it could even calibrate your configurations and tweak your files to get the best possible result for your computer according to your options, but allow you to still make manual adjustments both to the tweaks and to the mods options.

I believe it to be and a system like that could be adapted to work not only with Skyrim but, just maybe, with any game or simulation and even make mods of one game compatible with another, and probably even create mods of its own. Maybe it could even make a game compatible with a completely different system, so you'd be able to play just about anything modified with any mod in your smart watch, smart glass, smart mirror and even give it a chance at your smart toaster. And when you consider the existence and possibility of "Cloud Gaming" you know I'm not kidding when I say that you could even play it on a device that has far less processing power for the game at hand with no lag depending on your internet connection.

The thing is, with a system like that, no one would have problems of trying to make the mods they wish compatible with each other and no creator would have to create compatibility patches for users as they complain. No one would face game crashing bugs anymore and any mod you created would be instantly bug free, maybe the system could even tell you where the bugs were, so you can clean it at your original mod. No more crashing, no more losing a 100 hours game thanks to it suddenly having CTD after CTD for no apparent reason. No more corrupt saves. No more bug fixing and no more compatibility problems.

 

[montky]

SCORE!

this is an intriguing post...

Are you on CogWerks or OpenCog etc?

how is your script-fu in GIMP or FO4's geck?

what framework do you envisage your concept in, and is that compatible with the

limitations of Unity?

 

I was wondering a similar thing in

"talking heads in FO4?" as AGIXML

2nd loop feedback of some kind...

so that users could input strings to the NPCs and have a discussion.

maybe, an array of MCMC or

schutzenberger-chomsky parsing...

we input a key-constraint for each 'character', and the player can input certain strings...

the NPC outputs are from near-matching

(or in the case of Takahashi, output=gibberish).

This back-end could learn from multiple users to respond to a pattern of user-inputs

as a catchall overflow exception - if a lot of users use particular slang a certain way, it can learn...

deep neural networks and such could be great for RPGapplications

 

though this idea is taken to 11 hehe.

its the game the alters itself hehe.

 

I think this is a great idea for

rapidprototyping and troubleshooting.

its also an interesting approach for 'more immersive characters', and 'games masters' etc during a game...

 

though, putting on a white-hat and systems analyst hat momentarily...

such things also represent a read-write vulnerability in a network...

so, a backdoor into a computer that way is also not desirable for most game users hehe.

 

many AGI and Deep Neural Network iterative processes I've yet worked with to date,

they cause many headaches... "is A subset of A*" type, change your OS partitions,

rot your machine kind of headaches...

they're great at repetitive tasks, or multiple steps in a short time,

and combinatorial tasks.

-----

 

thanks for sharing these idea and discussion -

I send a lot of question your way momentarily...

 

this train of thought, could lead to a lot of awesome things.

hopefully, more AGI-ers et cyberneticists team up as modder here -

don't be afraid to ask some question or discuss as you like it,

even these 'can it be plausible? can it be done?" hypotheticals -

hypothetical questions are great for brainstorming/SCAMPER-ing...

 

my mod fu in Unity or FO4 etc is not so great,

though if a lot of modders collaborate...

say, some from "This Week In Tech" + googlers + opencog + programmer etc,

it can be that awesome stuff happens.

[ElPolloAzul]

"Moonshot" is an unbecoming and muy mucho verdacht word these days, but:

These are not new aspirations, or even ones which are going unworked on. Legions of people have expended and are expending, right now, large chunks of their lives to realize small parts of this kind of dream.

This is just my very personal opinion, and certainly human beings have a great habit of being highly quotably wrong in just these sorts of instances, but...

I do think that the world will be a VERY different place by the time a true development like this comes about fully, authentically, in the way you describe. Don't worry: MOST of it is coming reasonably soon if people are up to putting in the time and the actually required anguish.

For what it's worth, I do really think that more widespread "mod concierge", package-management-on-steroids-type software is particularly close on the horizon. There isn't a heck of a lot stopping fans from creating and using software which picks and automatically download, installs, consolidates, and even personalizes mods which seem to be to the player's previously revealed tastes. [Note, automated patching and modifying and differencing systems do not constitute some sort of intellectual property loophole - out courts ignore many logical contradictions in their decisions.] User-friendly recommender systems which can robustly diagnose and even attempt to solve different non-trivial types of mod incompatibility are already very realistically within our grasp. The only kind of factors stopping that kind of thing going wild are that there is for a number of reasons, no clear profit motive to a kind of software project that intensive and yet that specific to a community - once mod creating and consumption becomes a more inescapably mainstream part of gaming, that's a software need that people are going to fulfill for notoriety, practice, or just a warm fuzzy feeling. And surely, quite a lot of software incompatibility detection and remediation software will come out in general - and some of it will have to rely upon machine learning, even the newly attractive representation learning type where you don't have to pay a bunch of people to sit around and think up new features or transforms for years at a time. In general, automatic software fault detection, self-testing and "self-healing" and error-correcting software are big niche fields in academic software engineering these days and there are already some number of more promising startups in this area, of a new wave which embraces the fashionable machine learning techniques (c.f. Cylance in the computer security / antivirus world). [And, of course, streaming gaming is currently a very workable thing in some geographies, despite many peoples' personal experiences with high-latency, low-fidelity compression Remote Desktop / VNC kinds of things.]

But there are a lot of techniques to accomplish these sorts of things a lot better than we do today, and it isn't clear that a deep neural network alone, rather than a complex ensemble of a lot of approaches, some cultivated, and some very rigidly engineered, is the best current tool for so ambitious a job.

Deep neural networks and their accomplishments over the last extremely several years are undeniably cool, and the field moves like lightning sped up a thousand times. That's coming from someone who lives and breathes this stuff from the computational neuroscience and AI perspective - they have rewritten the textbooks, literally, in the fields of computer vision, natural language processing, general theoretical machine learning, and neuromorphic engineering. Deep neural networks plastered all over academic papers and news articles today synthesize speech at the individual sample level, colorize and more gracefully upscale old black-and-white movies from texture alone, recognize e.g. different species of dog at multiple angles and in terrible and unlikely pictures of the same and can segment them out of a picture accurately without human help, they infer sounds from videos, help to optimize compression and cryptography algorithms, reconstruct maps for physically-based rendering from a few exposures, predict human saliency maps close to those derived from experiment and even from other neural network experiments which build classifiers from human EEG/EMG/MEG/MRI/PET/etc. data and non-human-primate cellular recordings, learn motor and sensory routines for simulated and real neurally-inspired robots and swarms of the same, diagnose diseases, help improve crop yields, help deliver ads, predict sales, and discover patterns in high dimensional data, assess threats and avoid obstacles in the physical world, create new products, predict aging and weathering, caption videos and translate between languages, and they even "write" (compose and assess) traditional programs, and assess, compete, and communicate with other deep networks.

It's worth pointing out that "computational creativity" uses of these networks are frequently augmented by using other sub-symbolic, computational intelligence AI techniques also founded on "gradient-free" stochastic optimization, like genetic algorithms, swarm and cultural computing, etc.

But the excitement in the networks (combined with more willingness to pour computing power, electrical power, enthusiast and student interest, and money into making them bigger and bigger, and more pervasive) is what's especially new. Deep networks are deeper than before (and more of them are convolutional, looking at input on multiple scales and combining processing in a sensible hierarchy), but neural networks research has been around and somewhat productive, off and on, for a while (think since the 1950s if not actually somewhat earlier). We are in an early age of discovering useful network topologies to compute with, beyond the very well established layered, feedforward networks of artificial rate neurons we call multilayer perceptrons and train (for now) with algorithms like backpropagation. We both understand deeply and do not understand at all the full extent of the mathematics behind and computational power within these structures (recent times have seen the beginning of the demise of obligate high-precision inference, sigmoid functions, and gradient-descent). That said, many deep neural networks (with notable, but more specialist-oriented exceptions) on their own are fancy, compressed ways to implement a phonebook or owner's guide - they are chiefly used, albeit very successfully, to perform regression and classification tasks. While there is a lot of promise in designing nets for other tasks, the mania today is in arenas of detection and action selection - thankfully, those two things encompass a whole lot!

Computers already design (and particularly evolve) their own visual art and even their own video games, films, and written stories to some extent (mostly evincing a combinatorial, uninspiring form of creativity). Artistic style transfer and image analogies and deep dreaming and "AI neuroscience" research is similarly very freaking cool, but when you understand the mathematics behind it to the reductive extent of max pooling and strides and Gram matrices, the magic fades. Optimization and selection are powerful computational processes. But it is a subtler art that few computer models even today feign to possess to cater to arbitrary and unvoiced whims by creating new video game modifications and stories which are compelling, individuated, complex, and built on lifetimes of deep rather than exclusively broad knowledge. And while there is a lot more research than the average person thinks into making computers which write their own intricate programs (doing, in the process, much more than just very commonplace code generation) we are not near the point where we can have a system decide to design its own game, market it and write critiques and defenses of it, port it across many incompatible systems (currently and some may say fortunately, we lack the robotics or other technologies to allow these systems to physically instantiate a bunch of objects according to their will), do creative maintenance well into the product's future to preserve it when old systems fail, and perform general knowledge, par-human social and creative tasks at the same time with aplomb. Predicting all errors and fixing them perfectly, as in avoiding software faults completely, is on the order of completely impossible. On the other hand, some number of us alive today will unavoidably die in humanity's totally justified race to breed and train systems rather than program them - no system can know all of what it doesn't know, and there will be blind spots where unpredictable, stupid decisions are made - the same is true of rule-based systems in a domain with complex enough dynamics. In other words, with complex enough expectations, it is entirely *impossible* "to create a Deep Neural Network that is able to bug-check and fix any problems". You, or somebody someday, will be disappointed somehow - very likely, even in a domain as circumscribed as you describe.

We currently have some very fancy, extremely deep and intricate neural network powered machines capable of many if not the full complement of these wish-fulfillment functions - they are called humans, and certainly if you entertain a small set of fundamental cosmological beliefs, it is possible to eventually artificially pull off all of the things we do with a sufficiently advanced understanding of all of "neural networks". At present, though, we have such a laughably small understanding of their potential intricacy that some of us think that having a tiny zoo of network shapes, some differential equations, and an army of willing oscilloscope and fancier equipment monitors has got us most of the way there. The neural networks we have today, deep or no, do a lot of very impressive things, but in the grand scheme of it all our command of them, and certainly our understanding of their current and ultimate limitations, does not impress.

[HandofBobb]

I'd be concerned that a DNN would decide that Dogmeat needs to be a giant 6-eyed giant wolfspider that pukes radiation.

And that would be one of the 'functional' mods- not to mention all the failed attempts that it would come up with bogging down the game.

[Runestones]

I'd be concerned that a DNN would decide that Dogmeat needs to be a giant 6-eyed giant wolfspider that pukes radiation.

And that would be one of the 'functional' mods- not to mention all the failed attempts that it would come up with bogging down the game.

Haha, I believe that wouldn't quite happen ^^'' unless you had a mod for that to happen, then it would XD

The failed attempts wouldn't happen on your computer, so I believe that to not be that much of an issue

 

-------------------------------------------

 

I have to thank both ElPolloAzul and Montky a lot for their replies and interest demonstrated.

To be truthful, I really am not that knowledgeable in this field, so I basically grasping at straws and using of suppositions with what I know about DNN's to create this idea.

 

My background is: I'm an amateur futurologist, still trying to get into college to learn Control and Automation Engineering. I have a deeeeeep love for technology, innovation and new techniques, so I always try to be up on the latest and greatest and that which I find fascinating, so I have come across results from works with DNN's from time to time, and I do have a wish to learn and know more about them to apply it to my works if and when I finish the college I haven't even started yet.

 

I just wished to share this idea I had with the DNN's that I understand just superficially right now to see if it is possible of applying it to those applications mentioned and possibly more, as I believe they could also work to port mechanics and mods from one simulation to another of any kind after a while, helping not only game industries and modding communities, but also any company that uses simulations as well, such as simulated engineering which, if someone in the area of games develops a mod that creates an almost perfect physics to a point that it is astounding, they could also por such physics to their own simulation program which would help a lot.

 

As I said, I'm not very knowledgeable, but I do am quite creative and able to see connections where some people some times miss them. I was always good at "connecting the dots" haha ^^'

 

I really hope it does interests people and would definitely help with whatever and however much I can.

I hope people didn't quite think that I wish to start or lead such a project as I believe I do not have even the minimum amount of knowledge to be in such a position, but I do wish to help it get started, if it already isn't, and help, even if its just help organizing the ideas a bit or thinking up ways to fund such a project (which I personally would choose crowd-funding, as it is a community driven'n'directed project XD).

 

Anyway, thank you all very much for your interest and for your replies, which where very interesting and filled with pertinent information which I'll probably read a few more times until I can better understand that which I didn't quite get at first as there is a LOT of info there, but please, feel free to share this idea as you wish as well and contact me with whatever I can help to make it a reality as well. What I wish is for this to possibly become a reality and will do what I can for that.

[ElPolloAzul]

Controls & Automation Engineering is a fantastic place to be for many of the really societally useful application areas of this type of stuff; mainline "deep learning" research goes on in Computer Science, Electrical Engineering, Statistics, and Cognitive Science and Neuroscience departments as well. Keep in mind there are a few kinds of DNN (although most exciting ones today are feedforward convolutional neural networks with some fully-connected MLP "inference layers" at the end, or larger networks which use these as intermediate superunits).

Some more artistically oriented folks have put together an incomplete bestiary of popular topologies (http://www.asimovinstitute.org/neural-network-zoo/).

If you want to understand the mathematical prerequisites (multi-dimensional calculus, advanced linear algebra, mathematical optimization, and graph theory) for what we currently believe is "really going on", most machine learning and computational intelligence textbooks have garbage coverage of neural networks (the quality of teaching in this field is also usually not good unless you are at a really world-class university). The physically-forthcoming book by Goodfellow, Bengio, and Courville (http://www.deeplearningbook.org/) is (one of?) the best yet for getting up to speed without months or years of floundering with incomplete treatments or "learning by braille" (bootstrapping your way up by looking at papers).

To really look at the cutting edge, take a gander at the accepted papers at, e.g. NIPS and CVPR. If you have a higher tolerance for stepping in some incorrect work, take a look at the many papers daily which are preprint-served at the ArXiv, in the Neural & Evolutionary Computing Section.

 

If you want to look at the most widely read general machine learning textbook, many people start with Bishop's PRML, a more up-to-date data dump which crams as much coverage in as it can pell-mell is Murphy's A Probabilistic Perspective. If you are into computer vision (a good discipline for industrial automation) the leading books are highly varied in quality -- my advice is to go with Szeliski (as with the Deep Learning book, the full text is online legitimately and for free, but you will cherish a hardcover copy).

Finally, if you have several more contiguous hours to spare, watch the YouTube recordings of the CS231N (Convnets) and CS224D (NLP with Deep Nets) courses at Stanford this past year, both are taught by wunderkind recently-former-PhD students of the same institution. It's worth noting that high-profile research here (as in many fields, really) is an elite, rich-get-richer kind of phenomenon in pedigree and resource and assistance access terms.


Neural-net driven software configuration and also "computational engineering" in the sense of automated design optimization for all kinds of products (circuits, toys, antennae, etc.) are reasonably new, very promising specializations. Physics systems in games are bad for sociological reasons (game developers aren't usually the best and broadest physics students) but also "ludological" reasons (bad physics is fun to play with) and computational reasons (doing a full rundown of all of just ordinary mechanics at all relevant scales is expensive on current computing architectures). You will want to get that specific (or better) if you are headed to graduate school eventually - kind of a prerequisite for directing projects in established industries; you will also want to think very reductively and define your inputs and outputs very concretely and plan for scale of protoyping ideas, being first a good programmer will tend to constrain your ambition more instinctually.

 

Lately, there is no shortage of people who specialize in throwing money at this problem and organizing funds, hyping big ideas, etc. That you really can't compete with, in the main.

Edited October 30, 2016 by ElPolloAzul