WorldSpawn Texture Atlas

[MadGypsy]

As some of you may have read in my "real flash quake" thread I am building a texture atlas app to aid me in game development for my engine. My app will spit out a data file that accompanies the texture atlas. This data file will contain the UV offset of each image within the atlas.

This is where I have a chance to give a little back to Spike and Baker. I can format the data file for my atlas any damn way you guys want. I can even spit it out as if it is a file of structs data, XML, csv's (lol)...anything.

If either of you have interest in this, all you have to do is tell me how you want the file structured. I mean literally show me the structs you want me to use and any other markers that you would need in the file. For instance I believe animated textures should come as a set within the data file so, at the beginning of each entry there should be a byte that is checked for entry vs animated set. IMO animated sets should contain the min/max UVs for each image in the set and of course at least the name of the first image in the animation.

I can get tricky with this too. For instance, with some clever atlas construction [n]'s right can become [n+1]'s left, pretty much cutting the data in half.

I don't know if you guys have any use for an external texture atlas system but, if you do, I'm building it right now and I'm putting the c version of accompanying atlas data in your hands, your decision entirely on how it is structured. You tell me. Get together and create the "standard"... whatever makes you happy. Y'all have time. I am going to build this up completely to my needs (ie export as AMF2 serialized bytes) before I even think about other export methods. This app is going to take a "minute" so, there is no rush on anything you guys want me to do with this.

With enough premade atlases you could pretty much dump reading mips altogether or at least look in atlases first. A BSP file could be cut down greatly in size by simply naming a bunch of 2x2 colored squares after the real textures in the actual map so, upon compile the mips are almost non existent but, point right back to the actual texture in the atlas. Of course that would break in any engine not using the atlas system but, it doesn't have to be mandatory... more like an option for mappers. Actually why don't we make a new BSP that simply has the atlas info where miptex_t is? I bet you 100 bucks that I'm gonna do that regardless.


//other idea for the future
completely dump lightmap data - use entities_t to remake all the lights real time with 2 possibilities -
poss. 1) stay realtime
poss. 2) blink the data to a leafwide texture and remove the light

[MadGypsy]

real replacement textures

ie...not this


oh whoops I had my other display on, sorry so big...on the pic.

side thought: QRP.pk3 is over 400mb, the exact same images converted to atf format (which is already compressed) is about 70mb. Also the images in atf are supposedly designed for specific gpus. Currently I am using mac and win.

edit: The first-visit-geometry-stutters are completely gone since I switched to this texture format and I haven't even started merging faces yet.

[MadGypsy]

I wanted to convert the entire qrp to atf. That's almost 700 textures. Which means I would have to write almost 700 lines of this (png2atf -c d -r -q 0 -f 0 -i ptextures/wood1_7.png -o gameTex/wood1_7.atf) or I could be real smart and write a dynamic version of that line 1 time. Accompanied by a multi file selector that not only assigns the dynamics but does it in a loop for every file. Whereas that may sound complicated to some, you can see in my code there are only like 5 lines that are doing anything. That's my current document class, ie..that's all the code. I mean obviously the packages I imported are doing a bunch but, I didn't have to write that code.


However, running that command line is taking a loooong time. It's been about 10 minutes and it's finally on the c's.

[MadGypsy]

Yesterday was a backwards day. I spent much of the day making something that is not what I should have been making, due to confusion (I suffer from this...lol). Today is a wiser day.

I need spikes solution. I also need to merge meshes, there is no way around this. I have told my map not to draw any lightmaps... be fullbright, and there is still a subtle stutter when first visiting semi-dense geometry. My next step is to turn my parser into a re-compiler cause the next steps shouldn't have to happen every time the game starts.

1) build a fat pvs
2) merge all faces of the same texture (breaking them from the current mesh)
3) gather all new geometry,uv's,etc from all combined face meshes and rebuild into a new mesh
4) save that mesh data

some a)'s and b)'s in there would be to take the original images and convert them to ATF, ignoring bsp miptex_t altogether and maybe even dump lightmaps entirely. I need to run some tests but I'm pretty confident I could "bake" rtls across entire meshes as soon as the leaf is created or even just use rtls.

@ bake - maybe even save that lightmap as one big texture or maybe even smaller textures within an atlas...in ATF format.

Oh, I noticed something a little while back. The only maps that overflow the lightmap length, in my engine, are those that have colored lights.

[MadGypsy]

I found something that will bring me a step closer to better performance. It is specifically designed for flash.
Introduction to ATF Textures | Away3D Tutorials Wiki

I'm sure that tech, plus what spike said, plus atlasing lightmaps is going to make some big differences. Just as a note, the performance of my engine isn't terrible. I just want to get rid of the "first visit stutters".

[Baker]

If it is Quake and you want it to work with anything from DOSQuake/WinQuake to GLQuake to DarkPlaces ...

The specs are here:

Making Textures For Quake

Written by metlslime

But the short version:

"All texture dimensions must be multiples of 16."

Except liquid (must be 64x64) and sky 256 x 128.

[Adam]

Apologies for asking a slightly off-topic question but seeing as all you guys are here and are talking about texture sizes...

If I am making a texture set to use in a project, utilising a modern Quake engine, what is the most efficient size to use?

Taking into account that the artwork will be low resolution pixel art, so large sizes wouldn't be required.

Would it always be the smallest like 8x8, 16x16, 32x32 etc..
Or is it more the case that modern engines prefer larger sizes like 256x256 and upwards?

As a sidenote to this question, I always convert the Quake palette that I am using into a 32x32 texture set. It would be handy to know if that is the best size to use for this task, as resolution-wise they could be 1x1 for just a colour (I think 8x8 is a minimum tho', right?).

Also, this thread is interesting.
Is atlasing textures a process of collecting a set of smaller textures, like 32x32 and placing them into a much larger sheet (like a sprite sheet as mentioned by Spike) and is this more efficient in some way (or not as seems to be the case)?
If this process is beneficial, is there a nesting* algorithm that slots mismatched texture sizes into one sheet in the most efficient manner?

*(in case you use another term) Nesting being making the most efficient use a of a standard-sized sheet of supplied material, by moving the pieces required around, till you fit the most in.

[Baker]

Detail oriented boredom ...

npot has been standard fare since gl2.
quite frankly, engines that still don't use it are laughably obsolete.

Mark V uses npot and has for almost 2 years. Even on model textures (Quakespasm doesn't AFAIK because FitzQuake 0.85's gl_mesh.c was written to output padded coordinates and would be a bit of work to reverse out.).

Edit = yeah, Quakespasm doesn't npot model textures Current code for gl_mesh.c

It's also part of the reason Quakespasm/FitzQuake 0.85 can't do external textures for .mdl models.

void GL_MakeAliasModelDisplayLists (qmodel_t *m, aliashdr_t *hdr)
{
	int		i, j;
	int			*cmds;
	trivertx_t	*verts;
	float	hscale, vscale; //johnfitz -- padded skins
	int		count; //johnfitz -- precompute texcoords for padded skins
	int		*loadcmds; //johnfitz

	//johnfitz -- padded skins
	hscale = (float)hdr->skinwidth/(float)TexMgr_PadConditional(hdr->skinwidth); <---- here
	vscale = (float)hdr->skinheight/(float)TexMgr_PadConditional(hdr->skinheight);
	//johnfitz

even gles2 guarentees support for some npot, on the condition that you use clamp-to-edge and no mipmaps.

^^ That. And it's annoying.

[MadGypsy]

Your words were definitely not glossed over, nor are they gibberish but, they take some processing. Trying to shove your knowledge into my brain comes with effort. However, I do think I have just about parsed your script. Give me a cigarette to think about it, I'll realize what the one thing that is not obvious is actually pointing to.

[Baker]

@And your plan is, I guess, resize every texture into a square
no, my plan was to exactly paint the textures into a larger image at their original size wasting as little to no room as possible.

So I have a 64 x 64 image, a 128x32 image and 16x16 image.

And you aren't going to resize them and will instead use a tile size of 128x128 in that scenario, wasting all kinds of space for the 16x16 image?

So your effort to save space will waste tons of it.

(And there is what Spike said about needing to repeat a texture twice to avoid unwanted consequences ... you can stamp a big "TIMES 4" on that)

/Anyway, I suppose all the potential logs have been thrown on this fire. Just remember, if discovering a new idea is a hammer --- it doesn't make everything a nail just because you found a new hammer.

[Spike]

So what do I do? Are there any suggestions on a sane way to have all the faces in a leaf sharing ultimately the same texture? If not, I will need to invent one.

did you even read my remarks about efficient batching? or did you just gloss over it?

if my words are gibberish to you, lets try code:

void WalkLeafs(unsigned char *pvs)
{
	int l;
	struct rleaf_s *leaf;
	for (leaf = leaves + submodels->visleafs, l = submodels->visleafs; l-- > 0 ; leaf--)
	{
		//leaf 0 is not in pvs.
		if (pvs[l>>3] & (1u<<(l&7)))
		{
			for (unsigned int i = 0; i < leaf->numsurfs; i++)
			{	//a surface can be present in multiple leafs (when it crosses a node), so avoid adding the same surface multiple times.
				msurf_t *s = leaf->surf[i];
				if (s->markframe != markframe)
				{
					s->markframe = markframe;
					for (unsigned int j = 0; j < s->numidx; j++)
						*batch_indexes[s->batchid]++ = s->firstidx[j];
				}
			}
		}
	}
}

maybe you can somehow figure out what's actually going on there...
side note: this only really applies to the world. doors and ammo boxes and stuff can just have static pre-computed index lists.
and yes, you can cache the index list until the viewleaf changes.


simply put, per-leaf meshes is stupid. many leaves contain only 1 surface, for instance. which results in terrible performance.

[MadGypsy]

So what do I do? Are there any suggestions on a sane way to have all the faces in a leaf sharing ultimately the same texture? If not, I will need to invent one.

@And your plan is, I guess, resize every texture into a square
no, my plan was to exactly paint the textures into a larger image at their original size wasting as little to no room as possible. The images don't even have to conform to power of 2 cause the parent image will.

I'm not saying my way is right or anything. I'm just saying what my way was going to be.

@spike - "clip" repeat uv's
I was thinking min(u|v) + ( (u|v%1) * ( max(u|v) - min(u|v) ) ) from atlas would take care of a lot of it (not meaning anything bitwise)

[Baker]

if by mainstream, you mean geforce2-era gpus then sure...
16k has been the standard since dx11 gpu first appeared. while dx10 gpus might be limited to 8k.

*cough* mobile platforms */cough*

[Spike]

DarkPlaces probably uses the non-power of 2 extension.

npot has been standard fare since gl2.
quite frankly, engines that still don't use it are laughably obsolete.
still, I guess running on microsoft's 1.1 implementation is an interesting challenge. pointless, but interesting I guess.

even gles2 guarentees support for some npot, on the condition that you use clamp-to-edge and no mipmaps. which is perfectly viable for lightmaps or hud images. it can even be used well enough for mdl skins as well (at least vanilla skins - note that this also solves the flood-fill glitches, although high-res replacements would need to be npot+mipmapped, that said the engine could easily enough just implement lod instead).
but yeah, full npot is needed for world textures, which isn't guarenteed on mobile.

[Spike]

B) GLQuake has never used those at all, nor any GL Quake engine including DarkPlaces or FTE.

FTE utilises the provided mipmaps.
Yes, this causes issues when people generated dodgy mips, but such issues are exactly shared by software renderers.

/Protip: You know many mainstream OpenGL graphics cards have a max texture width/max texture height of 2048 right? Your approach is exceeding likely to fail.

if by mainstream, you mean geforce2-era gpus then sure...
16k has been the standard since dx11 gpu first appeared. while dx10 gpus might be limited to 8k.
of course, if you're tailoring your atlases to the computer in question by doing it at load time instead of preprocessing it, then you get the best of both worlds.