New creature project

[Arcaen]

I've started playing the game again after many many years and was pleasantly surprised with all the progress, the game feels much more accesible now than it was before and I feel like taking a new project to upgrade it. I am considering in developing a new creature, but first I am testing the game to check if it's even possible at all. During the last few days I've discovered the following:

-There are 31 creatures hard-coded into the game, that means no new ones can be added (and won't be possible in the future without extensive work, possibly never) replacing the existing ones remains the only option, but that doesn't make much sense considering the necessary effort.

-One of the creatures "FLOATING_SPIRIT" is actually just a placeholder to allow possession by the keeper once a level has been lost. I replaced this creature in the configuration file and it worked fine, the drawback is that the game now crashes if you try to walk around in first person after being defeated (This appears constantly in the log file: Warning: get_players_spectator_model: There is no spectator breed for player 0!). I decided to go down this route since It seems like a low price to pay to me, maybe in a later release of KeeperFX that feature can be disabled to prevent crashes.

-I've managed to get the creature name referenced in the gtext files and translated, but only by using the string reserved for "FLOATING_SPIRIT", there seems to be no way of adding new strings for naming things and making the game read them.

-Then I started to investigate the sprites and the animations, Keeper FX uses .PNG files, simple enough, but the formating in the filelist_rot.txt files is driving me crazy. It looks like this:

animation_xxxx A SS -DD -FF
frame1.png
frame2.png
frame3.png
frame4.png
frame5.png

-The xxxx number seems to be just the place in order of the line string that animation occupies inside the animlist.txt file, I've tried specifying a different name format and there seems to be no problem at all. The "A" number, I don't know what it does yet, it is usually 0 or 2, I suspect that 0 is for static sprites and 2 for dynamic ones. "SS" is the number of frames per animation. The last two numbers I have no idea of what they mean, maybe sprite scaling values in X and Y axis, but haven't had time to try them yet.

-The animations are referenced in the creature.cfg file in a funny way, it seems to be that the number the actual animation starts is not the one specified, but the one immediately before, as in if anim number 500 is the first one of the imp, and the 499 is the last of knight, writting 499 will show the imp, and not the knight.

-The number of sprites seems to be capped at 982, anytime I try to add a new one at the end of animlist.txt it gives an error in the log files (I am looking for it but lost it, it says something about an array overflow).

-I've run a few test with my own creatures.jpy files with different versions. Firstly I made my own version of an already existing creature animation and asigned it to my new creature. Since I failed to add it as a new sprite, I chose to overwrite one of the placeholder ones in the middle of the list, which had a single sub-animation with a single frame, with my modified walking animation. This had little success, on one side, the animation did show ingame, but it was only a frozen frame and I completely messed up the rest of the animations that were referenced further down in the animlist.txt file. Then I replaced the last one in the list instead and made a new creature.jpy file, this seems to work better, but still the sprite is frozen and also cropped in a weird way. I've also noticed this popping up constantly in the log files:

Warning: LbSpriteSetScalingData: Tried scaling width 66 -> 0
Warning: LbSpriteSetScalingData: Tried scaling height 27 -> 0
Warning: LbSpriteSetScalingData: Tried scaling width 64 -> 0
Warning: LbSpriteSetScalingData: Tried scaling height 27 -> 0
[etc...]

-I have also tried different image sizes to no avail, since I read somewhere that the limit is 250x250 or something like that.

So this has been it so far, I'd appreciate some comments and some help my way if possible, although I am very aware I might be hitting a wall real soon.

[YourMaster]

Welcome. Why don't you try to replace one of the regular units first? At least you'll know that works.
Also, does this help?

I have in fact managed to add gui strings to the game before.

And if you have the artistic talent to make new units, you could also look at the placeholder sprites that are already in the game and see if you can make something cool for that. There are - for example - a few placeholder statues I believe which just require a set of new PNG files to be used. Currently all decals are designed as evil, like demonic statues and what not. Making a statue that looks like it would fit in a hero dungeon would be a nice addition already.

And which animlist.txt are you looking at? There are much bigger ones than you mention, like GFX\sprites-32\animlist.txt.

[Arcaen]

Welcome. Why don't you try to replace one of the regular units first? At least you'll know that works.

That should work I think, as long as I replace all the correct files. The thing is I already like the creatures that are in the game, I don't want to have to replace an existing one to introduce my own (which I doubt to be more valuable than the one replaced), so in the end it doesn't really matter to me whether this works or not.

Also, does this help?

I am afraid I'm not the programmer type, this is beyond my abilities, I am some sort of graphic artist.

I have in fact managed to add gui strings to the game before.

Excellent, I was doing it wrong then. My method was editing the gtext_language.pot file and adding a line at the end with the flag corresponding to creature name definition and then putting the correct NameTextID line reference in the character.cfg file, but ingame it only showed an empty square as the creature's name.

What I understand from your link (correct me if I'm wrong) is that adding a new string requires to modify the config_strings.h file, and then recompiling the game (which is completely outside my knowledge). I understand that a lot more can be done from the source files rather than the game's folder files.

And if you have the artistic talent to make new units, you could also look at the placeholder sprites that are already in the game and see if you can make something cool for that. There are - for example - a few placeholder statues I believe which just require a set of new PNG files to be used. Currently all decals are designed as evil, like demonic statues and what not. Making a statue that looks like it would fit in a hero dungeon would be a nice addition already.

That would be a good start, that is an easier milestone and I could test whether I can produce graphics similar to the game or not. My current plan is to create a 3d model and then generating the frames from it rather than drawing by hand, although I estimate a statue would take only a minimum of 10 frames to achieve.

And which animlist.txt are you looking at? There are much bigger ones than you mention, like GFX\sprites-32\animlist.txt.

I am looking at that one, in my file there are 2268 lines, the last animation is "furniture/flagpole_whiteflag_tp/filelist_rot1.txt" and the animation count goes up to 982 (which is also specified as the animation's name, for example "magic_dust/anim0982/filelist_rot1.txt"), tell me if I'm correct.

[YourMaster]

That should work I think, as long as I replace all the correct files. The thing is I already like the creatures that are in the game, I don't want to have to replace an existing one to introduce my own (which I doubt to be more valuable than the one replaced), so in the end it doesn't really matter to me whether this works or not.

I understand you'd want the original units back, but you'd have a unit you'd be able to show off, and perhaps somebody who can program can help with adding it to the game at a later point in time.

I am afraid I'm not the programmer type, this is beyond my abilities, I am some sort of graphic artist.

Look at line 76 and beyond, it's a list of sprite properties.

What I understand from your link (correct me if I'm wrong) is that adding a new string requires to modify the config_strings.h file, and then recompiling the game (which is completely outside my knowledge). I understand that a lot more can be done from the source files rather than the game's folder files.

Basically yes.
But I can, and am happy to help. I shared KeeperFX Unofficial on the board here, which is mostly fixes and improvements I could do without being able to program. And since then, some people who can actually program have offered to help and so there's some occasional further progress. See here. If you share a Unit that would be nice to add into the game I would be sure to include it, and adding a text string would be a minor thing.

That would be a good start, that is an easier milestone and I could test whether I can produce graphics similar to the game or not. My current plan is to create a 3d model and then generating the frames from it rather than drawing by hand, although I estimate a statue would take only a minimum of 10 frames to achieve.

Statues and other decals only have one side. And one more for possession. But great, if you need help or feedback, be sure to let me know.

I am looking at that one, in my file there are 2268 lines, the last animation is "furniture/flagpole_whiteflag_tp/filelist_rot1.txt" and the animation count goes up to 982 (which is also specified as the animation's name, for example "magic_dust/anim0982/filelist_rot1.txt"), tell me if I'm correct.

Ok, I was simply looking at the number of lines, but we're seeing the same thing now. I'll have to look into this.
EDIT: There's a limit of 982 defined here, it might me simply a case of raising that number by 1 (or 22 for a unit) and recompiling. Share your creature.jty through PM if you don't mind, I'll see if it works. If so I'll provide you with a prototype you can test your animations with.

EDIT2: I actually made a creature.jty myself, tried to raise the number of animations it would allow and fixed all the errors in the log, and as a result I no longer got an 'empty' animation but the 'default' walking hellhound. So that's a disappointment, and it would require somebody with more skill than me to either raise that limit or say how difficult it is to get it to work.
You do have '10' placeholder animations to work with, but that's just half a creature.

[Arcaen]

I've also tried raising the limit in creature_graphics.h myself, but I think it only gets rid of the error message in the log file, the animations I add don't show up ingame. There is also another mysterious parameter in engine_render.h at line 34, which coincidently is the total amount of .png files in the sprite folder, but haven't touched it yet

I edited one frame in the walking animation of the fly and found out this:

-Changing the color of pixels is fine.

-Even adding a single pixel surrounded by transparent ones is not fine, adding a pixel right next to others is cool (but only 1 or 2).

-Surprisingly, shifting rows of pixels around is fine (as long as the row is complete it doesn't matter where it starts or ends in the canvas, but it will look funny ingame of course).

-The game doesn't care if I place the image centered or offcentered inside the frame, it will always show correctly as long as the parameters in filelist_rot.txt are correct.

This has led me to believe that the images are stored as rows of pixels, and the game reads them all at once and creates a big ass file with all the frames in it. Then it uses some sort of bookmark to know where each animation starts and ends, and if I add or remove pixels in a given frame the bookmark fails, because I displaced all the pixels that come after it. That's why everytime I change something, all the rest of the sprites are drawn displaced, or even showing part of the next sprite. This also means to me that it's not currently possible to add higher resolution sprites, because unless the bookmarks can be changed, the total number of pixels is going to grow, and the game is going to keep looking for the same ones that are not there anymore in the big ass file. I guess Mefistotelis must have had a solution in mind for this, as he was planning to add bigger sprites and even in a different format.

The next thing I did was investigate how the bookmarks work, so I created an image with the same number of non-transparent pixels as the original one to see if it was counting a total amount of pixels, but this didn't work. I've done several tests more to see if it counts rows, columns..., the conclusion is that the game seems to count pixels, in rows from left to right, and not only the solid pixels, but also the transparent ones that might be in between. They are not interchangeable. As soon as the last solid pixel in the row appears it skips to the next one and keeps on counting. It seems to be like this but I refuse to believe every sprite is so hardcoded into the game.

Whenever I have time I'll try to look up more values in the source files, but the ones we've found seem promising.

EDIT: Also, now I know what the numbers in the filelist_rot.txt files mean. The first one is a flag to indicate if it's allowed to mirror the frames: 2= yes, 0=no. The two last ones are correction values for the X and Y axis position (in pixels maybe).

[YourMaster]

Yeah I already mentioned that when I raised the limit I got rid of the error message but was not able to add another animation to the end of the list.

I was however perfectly able to just put a new animation I made somewhere in the middle and assign it to a unit without issues, so I assume there's no hardcoded stuff.

[Arcaen]

Wow, could you explain to me how you did it? Did you use a placeholder spot?

[YourMaster]

Well, not even that. Because I was just testing for you, I added two animations(a single set) at the end. I made a duplicate of the guardpost flags and made it white since that one is missing from the game. Since I found that it didn't work, I just moved those two lines up a bit (breaking the last few animations). I assigned that animation to 'gotslapped' on the imp, and whenever I slapped an imp it turned into a white waving flag.

[Arcaen]

Okay, this is embarrassing. I just found out that there is also a .TAB file that needs to be generated! This is the file that tells the game all the "bookmarks" I was talking about, and all this time I've been using the original .TAB file with my .JPY file, hence all the errors...

I just tested it, with my .TAB file now there are no errors and everything shows correctly. Now it's time to try and raise that frame limit, if I don't succeed I'll start thinking about using those placeholder slots in the best way possible and see what I can do with them.

[YourMaster]

Ok, great. Progress. And I'm sure at some point Spartahawk or Orson will look into raising the limit as well, but no guarantee they'll succeed of course.

If not, or in the meantime, one other possible addition would be to add some of the missing torture animations. Most units can be tortured and show up in the torture room, but there are several - like the wizard - without any torture animation, in which case the 'default' one is used that does not show the actual unit anymore.
The challenge of course is that you don't have the original models of the original units.