Announcement

Collapse
No announcement yet.

WorldSpawn Texture Atlas

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • WorldSpawn Texture Atlas

    As some of you may have read in my "real flash quake" thread I am building a texture atlas app to aid me in game development for my engine. My app will spit out a data file that accompanies the texture atlas. This data file will contain the UV offset of each image within the atlas.

    This is where I have a chance to give a little back to Spike and Baker. I can format the data file for my atlas any damn way you guys want. I can even spit it out as if it is a file of structs data, XML, csv's (lol)...anything.

    If either of you have interest in this, all you have to do is tell me how you want the file structured. I mean literally show me the structs you want me to use and any other markers that you would need in the file. For instance I believe animated textures should come as a set within the data file so, at the beginning of each entry there should be a byte that is checked for entry vs animated set. IMO animated sets should contain the min/max UVs for each image in the set and of course at least the name of the first image in the animation.

    I can get tricky with this too. For instance, with some clever atlas construction [n]'s right can become [n+1]'s left, pretty much cutting the data in half.

    I don't know if you guys have any use for an external texture atlas system but, if you do, I'm building it right now and I'm putting the c version of accompanying atlas data in your hands, your decision entirely on how it is structured. You tell me. Get together and create the "standard"... whatever makes you happy. Y'all have time. I am going to build this up completely to my needs (ie export as AMF2 serialized bytes) before I even think about other export methods. This app is going to take a "minute" so, there is no rush on anything you guys want me to do with this.

    With enough premade atlases you could pretty much dump reading mips altogether or at least look in atlases first. A BSP file could be cut down greatly in size by simply naming a bunch of 2x2 colored squares after the real textures in the actual map so, upon compile the mips are almost non existent but, point right back to the actual texture in the atlas. Of course that would break in any engine not using the atlas system but, it doesn't have to be mandatory... more like an option for mappers. Actually why don't we make a new BSP that simply has the atlas info where miptex_t is? I bet you 100 bucks that I'm gonna do that regardless.


    //other idea for the future
    completely dump lightmap data - use entities_t to remake all the lights real time with 2 possibilities -
    poss. 1) stay realtime
    poss. 2) blink the data to a leafwide texture and remove the light
    Last edited by MadGypsy; 03-07-2016, 02:43 PM.
    http://www.nextgenquake.com

  • #2
    here's my first draft on compressing uv data across a texture atlas. Only UV combos which do not include a 0 or 1 need to be saved.


    the dots represent array position in a left to right top to bottom order. In this case we would have something like: float offsets[9][2]
    the decisions could be made based on min/max column/row and if you are none of that get real min/max

    The only thing to be aware of with this method is, every image in the texture atlas has to be the exact same size.

    read order
    min,min 0 / 0,1 / 1,2 / 2, max,min
    min,derived 0,3 / 0,4 / 1,5 / 2,5, max,derived
    min,derived 3,6 / 3,7 / 4,8 / 5,8, max,derived
    min,max 6 / 6,7 / 7,8 / 8, max,max
    Last edited by MadGypsy; 03-07-2016, 08:10 PM.
    http://www.nextgenquake.com

    Comment


    • #3
      why do I get the feeling that you're aproaching this wrongly?

      atlasing the repeating world textures is futile. its a massive waste of texture memory, especially if there are replacement textures etc. essentually you end up with something akin to megatexturing, but without all the compression stuff that makes it practical.

      atlasing the lightmaps is almost mandatory. the data is much smaller, and not mipmapped.

      there are different aproaches to atlasing. you can be all fancy and use a binary tree to keep track of potential regions, or you can just do it in a lazy row-based way or something like glquake does.
      you could also just pad upwards to a power-of-two.
      the largest lightmap image in a 'standard' quake bsp is 18*18. fte+dp allow up to 256*256 of course, resulting in less surfaces and thus is a potential speedup.
      Some Game Thing

      Comment


      • #4
        @why do I get the feeling that you're aproaching this wrongly?
        {unsure} cause... I... am?

        @atlasing the repeating world textures is futile.
        Well, what is this then? A google search of texture atlas in images brings up all kinds of stuff like this.

        If I don't atlas the repeating textures, what other textures are there? They're all repeating textures. Only lightmaps? If I don't find some method to take all the textures in a leaf to one texture then I will never be able to merge leaf faces. Merge only works if everything being merged shares the same texture.
        Last edited by MadGypsy; 03-07-2016, 11:17 PM.
        http://www.nextgenquake.com

        Comment


        • #5
          Originally posted by MadGypsy View Post
          A BSP file could be cut down greatly in size by simply naming a bunch of 2x2 colored squares after the real textures in the actual map so, upon compile the mips are almost non existent but, point right back to the actual texture in the atlas.
          Are you referring the mipmap textures, the smaller versions that follow the "real texture"?

          a) Only WinQuake uses those.
          B) GLQuake has never used those at all, nor any GL Quake engine including DarkPlaces or FTE. Open GL engines make the smaller mip textures in the engine by resizing the "real texture".

          Also: no reason to worry about .bsp size. Everyone zip archives their maps. And if hard drive space were an issue, you'd just use compression like .pk3 or .zip for the files laying around like Q3 or DarkPlaces/FTE. (In fact, an Android .apk is a zip. So some platforms, you are working with compressed stuff anyway).

          Besides, if you were going to worry about size, then .jpg (or dds) would be the choice. But Quake textures tend to be small like 32x32.

          32x32 is a really small texture to begin with. And it isn't like Quake textures use 32-bit color, they are 8 bit so that's about a 75% savings in size right there over an rgba format.

          / Just pointing out various stuff -- I know you are sorting out stuff
          Quakeone.com - Being exactly one-half good and one-half evil has advantages. When a portal opens to the antimatter universe, my opposite is just me with a goatee.

          So while you guys all have to fight your anti-matter counterparts, me and my evil twin will be drinking a beer laughing at you guys ...

          Comment


          • #6
            Originally posted by MadGypsy View Post
            @why do I get the feeling that you're aproaching this wrongly?
            {unsure} cause... I... am?
            Well, if you are making Minecraft than that's what Minecraft does.

            But if you are doing fps, what are you going to do when a texture is really wide and very thin.

            Sort of like the "Quake" texture in the start map.

            Are you going to make every texture in your sheet 1200x1200 if there is one texture with a crazy size.

            /Protip: You know many mainstream OpenGL graphics cards have a max texture width/max texture height of 2048 right? Your approach is exceeding likely to fail.
            Quakeone.com - Being exactly one-half good and one-half evil has advantages. When a portal opens to the antimatter universe, my opposite is just me with a goatee.

            So while you guys all have to fight your anti-matter counterparts, me and my evil twin will be drinking a beer laughing at you guys ...

            Comment


            • #7
              Well, actually the biggest texture I can make is 2048x2048.
              I was going to sort images into square, wide & long.. knock out all the square and then use a 2d bsp algorithm to fit wides and longs in the remaining space. I had ideas for handling beyond one atlas, as well.

              @Your approach is exceeding likely to fail.
              It's totally possible, bro. I don't think there is a computer alive in the world today that can't line up some pictures and organize them to optimize the space or divide (image.x|y + image.width|height) by stage.width|height to obtain a uv. However, if I am going to mix texture sizes I won't be able to use my compressed uv coordinates.

              Edit: I guess another way I could go about this is how I was doing it before mixed with some new. I can build the repeating textures out to the surface extents and store that. Then when I get to leaf construction I can paint all the face textures for the entire leaf to a new image and refigure uv's. This way each leaf will have a single texture and I can merge the faces. I have to get these faces merged, it's a huge performance boost. Like night and day performance. If I intend to keep my static light method, my lightmaps are already surface extents wide, they can be multiplied right onto the corresponding face texture. I can also make it to where this only has to ever happen once. When the textures are created they can be exported and everything from getting the original image to painting the leaf texture can be skipped. The only thing that concerns me is that this could result in hundreds of "leaf atlases" even if I got real creative and tried to use 2 leafs on an atlas the number could still be really high. There is also the possibility that one leaf could exceed an atlas, that would really suck.

              So what do I do? Are there any suggestions on a sane way to have all the faces in a leaf sharing ultimately the same texture?
              Last edited by MadGypsy; 03-08-2016, 12:05 AM.
              http://www.nextgenquake.com

              Comment


              • #8
                congratulations! you found a sprite-sheet that contains a non-repeating atlas!

                a repeating texture is any texture who's texture coords are not within the 0-1 range.
                if you want to atlas them anyway, this requires that you chop up the geometry so that each repeat can repeat properly again.
                and this then gives you issues with borders and linear filtering.
                and if you want to support mipmapping, again you have issues with borders, as higher mip levels start to bleed into one another.

                hence why you should not bother atlasing the repeating world textures. use a texture array or something and you can reduce most maps down to only 8 batches or so.
                texture arrays are awesome if you're willing to depend upon gl3+/d3d10.

                however, lightmaps are a different story. any coord outside the 0-1 range is invalid, making it non-repeating. its also sufficiently low res that mipmapping isn't needed. this trivially solves all issues that normally plague the use of atlases.

                hud images are another example of non-repeating images where mipmapping is not useful.
                Some Game Thing

                Comment


                • #9
                  Part 2

                  GLQuake resizes textures that are, say, 48x40, to be 64x64. 48 becomes the next highest power of 2. 40 becomes the next highest power of 2. Both of those are 64 (next highest power of 2 size)

                  Now the process of turning a 48x40 texture to 64x64 is going to be a slightly odd and messy resize with some stretching.

                  Which is why GLQuake doesn't look nice compared to FitzQuake or DarkPlaces, etc. Neither of which do that resize operation at all. Old FitzQuake did padding. DarkPlaces probably uses the non-power of 2 extension.

                  And your plan is, I guess, resize every texture into a square --- sort of like GLQuake did to stretch resize the textures.

                  /If you see what I am saying ... I'm just pointing out things that in the general body of knowledge amongst those that have worked with engines.
                  Quakeone.com - Being exactly one-half good and one-half evil has advantages. When a portal opens to the antimatter universe, my opposite is just me with a goatee.

                  So while you guys all have to fight your anti-matter counterparts, me and my evil twin will be drinking a beer laughing at you guys ...

                  Comment


                  • #10
                    Originally posted by Baker View Post
                    B) GLQuake has never used those at all, nor any GL Quake engine including DarkPlaces or FTE.
                    FTE utilises the provided mipmaps.
                    Yes, this causes issues when people generated dodgy mips, but such issues are exactly shared by software renderers.

                    /Protip: You know many mainstream OpenGL graphics cards have a max texture width/max texture height of 2048 right? Your approach is exceeding likely to fail.
                    if by mainstream, you mean geforce2-era gpus then sure...
                    16k has been the standard since dx11 gpu first appeared. while dx10 gpus might be limited to 8k.
                    of course, if you're tailoring your atlases to the computer in question by doing it at load time instead of preprocessing it, then you get the best of both worlds.
                    Some Game Thing

                    Comment


                    • #11
                      Originally posted by Baker View Post
                      DarkPlaces probably uses the non-power of 2 extension.
                      npot has been standard fare since gl2.
                      quite frankly, engines that still don't use it are laughably obsolete.
                      still, I guess running on microsoft's 1.1 implementation is an interesting challenge. pointless, but interesting I guess.

                      even gles2 guarentees support for some npot, on the condition that you use clamp-to-edge and no mipmaps. which is perfectly viable for lightmaps or hud images. it can even be used well enough for mdl skins as well (at least vanilla skins - note that this also solves the flood-fill glitches, although high-res replacements would need to be npot+mipmapped, that said the engine could easily enough just implement lod instead).
                      but yeah, full npot is needed for world textures, which isn't guarenteed on mobile.
                      Some Game Thing

                      Comment


                      • #12
                        Originally posted by Spike View Post
                        if by mainstream, you mean geforce2-era gpus then sure...
                        16k has been the standard since dx11 gpu first appeared. while dx10 gpus might be limited to 8k.
                        *cough* mobile platforms */cough*
                        Last edited by Baker; 03-08-2016, 12:28 AM.
                        Quakeone.com - Being exactly one-half good and one-half evil has advantages. When a portal opens to the antimatter universe, my opposite is just me with a goatee.

                        So while you guys all have to fight your anti-matter counterparts, me and my evil twin will be drinking a beer laughing at you guys ...

                        Comment


                        • #13
                          So what do I do? Are there any suggestions on a sane way to have all the faces in a leaf sharing ultimately the same texture? If not, I will need to invent one.

                          @And your plan is, I guess, resize every texture into a square
                          no, my plan was to exactly paint the textures into a larger image at their original size wasting as little to no room as possible. The images don't even have to conform to power of 2 cause the parent image will.

                          I'm not saying my way is right or anything. I'm just saying what my way was going to be.

                          @spike - "clip" repeat uv's
                          I was thinking min(u|v) + ( (u|v%1) * ( max(u|v) - min(u|v) ) ) from atlas would take care of a lot of it (not meaning anything bitwise)
                          Last edited by MadGypsy; 03-08-2016, 12:27 AM.
                          http://www.nextgenquake.com

                          Comment


                          • #14
                            Originally posted by MadGypsy View Post
                            So what do I do? Are there any suggestions on a sane way to have all the faces in a leaf sharing ultimately the same texture? If not, I will need to invent one.
                            did you even read my remarks about efficient batching? or did you just gloss over it?

                            if my words are gibberish to you, lets try code:
                            Code:
                            void WalkLeafs(unsigned char *pvs)
                            {
                            	int l;
                            	struct rleaf_s *leaf;
                            	for (leaf = leaves + submodels->visleafs, l = submodels->visleafs; l-- > 0 ; leaf--)
                            	{
                            		//leaf 0 is not in pvs.
                            		if (pvs[l>>3] & (1u<<(l&7)))
                            		{
                            			for (unsigned int i = 0; i < leaf->numsurfs; i++)
                            			{	//a surface can be present in multiple leafs (when it crosses a node), so avoid adding the same surface multiple times.
                            				msurf_t *s = leaf->surf[i];
                            				if (s->markframe != markframe)
                            				{
                            					s->markframe = markframe;
                            					for (unsigned int j = 0; j < s->numidx; j++)
                            						*batch_indexes[s->batchid]++ = s->firstidx[j];
                            				}
                            			}
                            		}
                            	}
                            }
                            maybe you can somehow figure out what's actually going on there...
                            side note: this only really applies to the world. doors and ammo boxes and stuff can just have static pre-computed index lists.
                            and yes, you can cache the index list until the viewleaf changes.


                            simply put, per-leaf meshes is stupid. many leaves contain only 1 surface, for instance. which results in terrible performance.
                            Some Game Thing

                            Comment


                            • #15
                              Originally posted by MadGypsy View Post
                              @And your plan is, I guess, resize every texture into a square
                              no, my plan was to exactly paint the textures into a larger image at their original size wasting as little to no room as possible.
                              So I have a 64 x 64 image, a 128x32 image and 16x16 image.

                              And you aren't going to resize them and will instead use a tile size of 128x128 in that scenario, wasting all kinds of space for the 16x16 image?

                              So your effort to save space will waste tons of it.

                              (And there is what Spike said about needing to repeat a texture twice to avoid unwanted consequences ... you can stamp a big "TIMES 4" on that)

                              /Anyway, I suppose all the potential logs have been thrown on this fire. Just remember, if discovering a new idea is a hammer --- it doesn't make everything a nail just because you found a new hammer.
                              Quakeone.com - Being exactly one-half good and one-half evil has advantages. When a portal opens to the antimatter universe, my opposite is just me with a goatee.

                              So while you guys all have to fight your anti-matter counterparts, me and my evil twin will be drinking a beer laughing at you guys ...

                              Comment

                              Working...
                              X