WorldSpawn official WIP thread

[SourceSkyBoxer]

I will tell fix for Map<String, Sometype>
@callback.call() in HaXe

Just use callback()

Because it throws error: "_callback has not arguments"
Just call _callback.bind([this, event]);
Compile success!!!

But you said about Map<String, type> Awww I forget that. Because I am using like shit example IMG[img[Std.is(byName).length] It means wrong. Your right.

I will fix again. Sorry you're right I don't use Vector just "Array".

I have already fixed and It doesn't show texture's name. I already tried... But no errors.haxe compiler has built successful.

You can download my worked and converted haxe versionIt is my file from Mega.nz -> my account is not criminally.

[MadGypsy]

Re: callback()

callback(event);

Sorry for not being clearer. I felt it was pretty obvious though that you need to stick arguments in a function that expects them. However, your way is fine too, maybe not optimal though.

@shit code
Blame flash bro. It's a workaround for slow ass dictionary. I could have just done it all with Object but then I would lose the strict typing of the Vector...do you see how it is what it is because it's fast and remains strict typed. I don't like it either but, it's either that or have a clunky library. All the weird syntax is only in Content.as so, it's not like any of the rest of your code has to manage it.

[SourceSkyBoxer]

Okay you said it:

Eh,_callback(event) sucks but it crashes Oh oh....
I try debug and listed:

Called from hxcpp::__hxcpp_main
Called from ApplicationMain::main ApplicationMain.hx line 79
Called from ApplicationMain::create ApplicationMain.hx line 111
Called from lime.app.Application::exec lime/app/Application.hx line 225
Called from lime._backend.native.NativeApplication::exec lime/_backend/native/NativeApplication.hx line 154
Called from lime._backend.native.NativeApplication::handleApplicationEvent lime/_backend/native/NativeApplication.hx line 195
Called from lime.app._Event_Int_Void::dispatch lime/_macros/EventMacro.hx line 101
Called from lime.system.ThreadPool::__update lime/system/ThreadPool.hx line 206
Called from lime.app._Event_Dynamic_Void::dispatch lime/_macros/EventMacro.hx line 101
Called from lime._backend.native.NativeHTTPRequest::threadPool_onComplete lime/_backend/native/NativeHTTPRequest.hx line 414
Called from lime.app.Promise_haxe_io_Bytes::complete lime/app/Promise.hx line 39
Called from lime.net._HTTPRequest_Bytes::load lime/net/HTTPRequest.hx line 124
Called from lime.app.Promise::complete lime/app/Promise.hx line 39
Called from openfl.net.URLLoader::load openfl/net/URLLoader.hx line 74
Called from openfl.events.EventDispatcher::dispatchEvent openfl/events/EventDispatcher.hx line 97
Called from openfl.events.EventDispatcher::__dispatchEvent openfl/events/EventDispatcher.hx line 218
Called from io.events.EventManager::event_handler io/events/EventManager.hx line 89
Called from openfl.events.EventDispatcher::dispatchEvent openfl/events/EventDispatcher.hx line 97
Called from openfl.events.EventDispatcher::__dispatchEvent openfl/events/EventDispatcher.hx line 218
Called from io.Importer::event_handler io/Importer.hx line 113
Called from Main::handle Main.hx line 42
Called from wad.WAD3::parse wad/WAD3.hx line 50
AL lib: (EE) alc_cleanup: 1 device not closed

Wow It throws unexpected errors.
openfl test cpp -debug -Ddebug than it shows unexpected errors on terminal or command or output of FlashDevelop.

[MadGypsy]

Are you using Function as your type? If so, that is wrong

_callback:Event->Void;

[SourceSkyBoxer]

Are you using Function as your type? If so, that is wrong

private var _callback:Event->Void;

I type this:

private var _callback:Dynamic->Void;

[MadGypsy]

That should work but, at the very least you should be...

_callbackpenfl.events.Event->Void

Seriously, forget Dynamic. About the ONLY time I can think that Dynamic is necessary is when capturing a Json.parse and even then you should back it with a typedef if possible. That way you can maintain some kind of strict typing.

Edit: Also, if you back your Dynamics with a typedef you can refer to any non-optional field directly (myDyn.someField) with confidence. This means you can skip Reflect for non-optional fields because, if the field didn't exist it would have already thrown an error claiming your Dynamic does not match the typedef. You can refer to optional fields directly, as well, but not with any confidence. At the least Reflect.hasField should be used before attempting to refer to any optional fields directly.

You however cannot key a Dynamic (myDyn[aKey]) and that is crippling. You would definitely have to use Reflect to achieve a key-like interaction BUT, there is another way. It just requires you to create an abstract for Dynamic and use the @:arrayAccess metadatas. I wrote the mother of abstracts for this when I was studying abstracts. All that said, Dynamic is still slow on static targets no matter what you do and should be completely avoided at all costs. Use Map or List. Speaking of List, you will not find a faster type but, there is a drawback. You can't use keys or fields on a List and have to instead loop through the iterator matching values til you get the one you want. Which means List is not an ideal container for most of your needs and you should probably just stick with Array<T>.

[MadGypsy]

Pbbt, I tried to share my Volume.hx script here and apparently my one class exceeds the character count...lol. Here's a link to it. I wouldn't recommend actually using it cause that means you are using Dynamic and you shouldn't be BUT, if you just have to use Dynamic due to there not being any other sane way to do something then dump it in Volume.hx instead. My abstract makes Everything act like everything. In other words, anything you can do to Map or Dynamic you can do to Map, Array AND Dynamic. Complete unification across the board* with a totally sick amount of custom options. I mean, the script is longer than the allowed character count of a post and it is optimized to death. It's crazy powerful and includes a novella of comments.

*obviously I can't unify Map or Dynamic to Array functions because how the hell would I handle push,pop,shift,etc... on containers that do not have ordered indexes...or any indexes for that matter? If you target an Array within Volume you can use the expected functions on that array though. That goes for any type. No type loses it's expected methods or values.

It does have one strange side effect though and I never could figure out a proper way around it (cause there is not one)

You can't
myVolume["field1"]["field2"]

You have to
myVolume[["field1","field2"]]
or
fields = ["field1","field2"]
myVolume[fields]

here is at least it's feature list and some explanation of typing so you have a gist before downloading

/**	Volume.hx
 * 		-Supported Types:
 * 		-Int, UInt, Float, Bool, String, Dynamic, Array, StringMap, IntMap, Enum, Volume
 * 
 * 		-Features:
 * 		-parse expressions
 * 			-automatically parse stringified math expressions, hex strings & hash strings to numeric results on key creation and assignation
 * 			-automatic expression parsing can be turned off with false for the constructors e argument - express() can be called later, if necessary
 * 			-if the e argument is true every assignment will reformat the new value
 * 		-reformat maps to dynamic
 * 			-automatically reformat all (String|Int)Maps to Dynamic. IntMaps are reformatted to dynamic with stringified numeric keys
 * 			-automatic reformat can be turned off with false for the constructors b argument - reformat(b, e) can be called later, if necessary
 * 			-if the b argument is true every assignment will reformat tne new value
 * 			-if reformat is called directly by the user _basic and/or _expr can be overridden in the arguments
 * 			-if reformat is set to true any container type can be used as the source data
 * 		-deletion
 * 			-if delete is performed with a null route this.data will be reset to an empty object
 * 			-deletion pop()'s the supplied route in place. if you passed the route by reference the reference will be pop()'ed ~ the field no longer exists so it is removed
 * 		-to string
 * 			-stringify object as pretty-printed|minimized valid json syntax
 * 			-stringify object as pretty-printed|minimized generic script object syntax
 * 		-built in looping
 * 			-inline forEach() loop ~ traverse immediate children
 * 			-inline forEvery() loop ~ traverse all descendants or just all dynamic descendants
 * 		-duplication
 * 			-get a unique copy of any container type
 * 		-defaulting 
 * 			-define fields to those of another object with the same type
 * 		-keying
 * 			-String|Int key access of immediate children/indexes ~ get/set/delete
 * 			-Array<String|Int|Dynamic> key as a route to any descendant or descended value get/set/delete
 * 
 * 		-FYI:
 * 			@route - 	
 * 				there are 2 route types:
 * 
 * 				route		- this would be any unprocessed route and can be of types Null, Int, String or Array<Int|String|Dynamic>.
 * 							  if route is null it will result in either the return of this.data or target:T as argument data
 * 
 * 				r:Route_t	- this is a processed route. All routes are converted to this internally.
 * 							  the raw route was unified with Array<Dynamic> and then either pop()'ed or shift()'ed into r.key (unless invalid... then null)
 * 							  the remaining unified route (or null) is assigned to r.route
 * 							  If pop()'ed r.route is the route to the parent of the r.key
 * 							  If shift()'ed r.route is the route to follow from the r.key
 * 							  
 * 			@containers -
 * 				container types consist of Dynamic, Array<T>, IntMap<T>, StringMap<T> and Volume<T>. the HxVolume value is used to determine if a value is a Volume<T>
 * 				it is NOT possible to chain keys even if the return value is an instance of Volume<T>. Use forEvery loop to have key access on descendants beyond immediate children
 * 				it IS possible to chain keys if the value of a keyed container is an array or map.
 * 				internal Volume<T> instances are temporary during for[Each/Every] loops. this allows child/descendant Dynamics to be keyed in the loop
 * 				all of the HxVolume cases are solely to handle the temporary Keys made by the built-in loops
 * 
 * 				@containers @example chaining keys for non-Dynamic containers (IntMap<T>,StringMap<T>,Array<T>)
 * 					var arr = ["one","two","three"];
 * 					var obj:Volume<Dynamic> = {cnt:arr};
 * 					trace(obj["cnt"][0]);			//"one"
 * 
 * 				@containers @example chaining keys for Dynamic containers	- WRONG WAY
 * 					var ent = {one:1, two:2, three:3};
 * 					var obj:Volume<Dynamic> = {cnt:ent};
 * 					trace(obj["cnt"]["one"]);		//Error: String used where Int expected for argument 'v'
 * 
 * 				@containers @example routing keys for Dynamic containers	- CORRECT WAY
 * 					var ent = {one:1, two:2, three:3};
 * 					var obj:Volume<Dynamic> = {cnt:ent};
 * 					trace(obj[["cnt", "one"]]);		//1
 * 
 * 				@containers @example routing keys for Dynamic containers	- ALTERNATE CORRECT WAY
 * 					var ent = {one:1, two:2, three:3};
 * 					var route = ["cnt", "one"];
 * 					var obj:Volume<Dynamic> = {cnt:ent};
 * 					trace(obj[route]);			//1
 * 
 * 			@constraints <T,D,K,R,V>
 * 				these 5 IDs are used to represent the types that are unknown from one recursion or call to the next of a method
 * 					@T 	(Type)	- a copy type
 * 					@D	(Data)	- type of the final parent in a route, also known as the target
 * 					@K	(Key)	- almost entirely used as an unknown for a map key. K should always be either an Int or String
 * 					@R	(Route)	- key multitype. either null, Int, String or Array<Int|String|Dynamic>
 * 					@V	(Value)	- the value type for user get|set related methods
 *
 * 			@null 
 * 				if you make a new Volume<T> with a null value for the source argument it will result in numeric key targets being treated as Dynamic
 * 				if you need a new Volume<T> instance to be treated as an array pass [] instead of null
 * 
 * 				@null @example null
 * 					var myKey:Volume<T> = new Volume(null);
 * 					myKey[110] = 20;			//any number can be used for the key
 * 					trace(myKey.toString()); 		//{"110":20}.
 * 				@null @example []
 * 					var myKey:Volume<T> = new Volume([]);
 * 					myKey[myKey.length] = 20;		//best way to get first available index if this.data is an array
 * 					trace(myKey.toString()); 		//[20].
 * 				
 * 			
 * @author	OneMadGypsy - 03/12/17 15:34:42Cn
 * @version 3.51.00
 *

side note: If is faster than Switch/Case in flash but, that probably is not true in haxe. I have not tested but, my Switch/Cases in this script use Enums, which are Algebraic Data Types, and the switches are long enough to qualify for jump tables. It is very probable that my switches are substantially faster than stringing a bunch of if/else if/else.

side note 2: If your Enum includes a constructor you have to use a switch/case whether you like it or not or there is no way to access the constructor . None of my Enums use constructors in these scripts.

Volume.hx has these 2 dependencies

My expression parser
Parse ANY stringified math expression to results, including every imaginable math function and/or every imaginable combination. 100% follows math precedence. This class uses a lot of tricky RegEx and is compatible with ALL platforms. It is not evil like Javascript eval. It only understands Math. It's pretty damn fast but, could probably be made faster. It seems to actually perform better with longer problems. Probably because there is some amount of milliseconds it will run even if you put nothing in it. Meaning a longer stack will have a better average per expression than a smaller one when divided into total time. Use double precision floats for best results. ie 10.50 not 10.5... Although, knowing me, I probably put something in there that makes your numbers double precision whether you like it or not in the condition where single precision is inserted...can't remember.

and

My custom type checking system
designed to understand ANYTHING you throw in Volume that is supported by Volume with NO exceptions and regardless if you ever strict typed it in the first place. If no strict type is provided it falls back on the results of type inference and makes it official. It would be very easy to extend this to accept more types but, it would not be very easy to extend Volume to use them. Aside from the fact that you would have to go to damn near every function in Volume to include the new type, you would also have to consider how the functions of Volume would affect the new type. A good example is List. How are you going to key a type that does not support any equivalent of a key? You would have to create some equivalent of a look-up table and then you might as well just use Dynamic. However, some form of a Byte type could be included but, it would probably be smarter to simply abstract a Byte type and only add @:arrayAccess getter/setter (if you need that) than to bloat the shit out of it with all the stuff Volume does. For instance, why would you ever need a ByteArray to be pretty printed to a Json object or converted to a Dynamic or like 90% of the other shit Volume does?

Put all of these scripts in the same namespace. All of this combined is one totally bad-ass example of how a lot of things work in HaXe.

FYI do not waste your time trying to figure out how to chain keys. I already know what needs to be done and it is a terrible solution. You would need to turn EVERYTHING into Volume, including Int, Uint, Float, String...everything. That solution is stupid as fuck but, it is the only one that will work. Because of that we DON'T do that and simply trade chainable keys for Array keys. Not that big of a deal.

There is another solution but it is even dumber. You would have to invent your own polymorphism by creating a Volume in every platform specific language that is supported and then use haxe to abstract it. Hardly seems worth it just to get a chainable key. Essentially, you would have to completely rewrite the entire Map type and ALL of it's connected material including all of it's platform specific code for every language. Pbbbbbt, No! lol

There is a third solution and it is one I intend to tackle in the near future. Simply convert all Dynamics and members to the proper Map type. Considering I would never in a million years use Dynamic unless I needed to capture some Json results, it seems like it would be smarter to create my own Json parser that never converts to Dynamic in the first place. If I'm going to do that I might as well extend it to parse "MapScript" (or whatever you want to call it) which would be little more than strict typed Json that allows for a hell of a lot more than the little stock QC equivalent of types. Anybody ever notice that? A Json object (not true Json) is like the QC of Javascript (Object, Array, Float, String, Bool...and that's it).

[MadGypsy]

I wanted to do a little test.

I have my wad parser parsing all 3116 images (plus mips) in halflife.wad and placing them in the library in 9.17 seconds.

I made a super small wad (85kb) and used TexMex to open it (as a control). It opened instantly (0.13s). I then used TexMex to open halflife.wad. It took over 36 seconds. The TexMex wad3 parser is approx 400% slower than my flash one. There is no contest.

Hmmm, I just realized there is one thing that is not fair about this test. TexMex is also displaying the images. I bet I could still do it way faster. Imma go see.

[MadGypsy]

pbbbt for me to display every image (all 3116) added a whopping half a second.

Yeah, still no contest. My parser is blazing fast.

[MadGypsy]

I just cut the time in half. My wad3 parser is now 800% (+/- a few percent) faster than TexMex. Wait til y'all see my BSP parser. My tests aren't fair yet because quake engines are doing more than my engine from execution to full game load but, considering what I have left and how much faster mine already is. My estimation is that I can go from execution to full game load roughly 2 times faster than any quake engine I have ever used, without utilizing a single worker.

My "trick" is that I know and only use the absolute fastest methods/types for my platform (currently flash) and my code never repeats itself, EVER. I also have no globals, at all. Nothing has been embeded, for those that might know and understand about that. This speed is achieved even though files are being loaded locally. If I embeded I could makee this even faster BUT be stuck with whatever I embeded bloating the app whether it was used or not.

Alternately, if I did use a worker, I could completely smoke a quake engine* due to being able to do more than one job simultaneously. I may look into that eventually. I messed with workers before and I was less than impressed with it's performance but, my tests were pretty stupid ones. I believe I was just running arbitrary calculations in a loop. Maybe with something more real-world to play with I would change my opinion and find it more useful.

*I say this because I think I already am and doing things simultaneously should make it all even faster.

[MadGypsy]

I realized I actually have another "trick". I write all my own parsers. Audio, image... everything. Almost all of them have also been optimized to death. Correct me if I am wrong but, at least most quake engines use some libraries that are already provided for them. Like ogg vorbis and other such things. Maybe (some of) those libraries aren't all that optimized. It's just a theory that came to me so, I have no idea how sound it is.

I'm only claiming to out (load/parse) these programs. After full (load/parse) is a whole nother story. I'm obviously not going to get framerates and stuff like that which real quake engines provide. I keep it pretty damn solid at my max though.

[MadGypsy]

Here is my final wad3 parser script. It's basically all in one loop and it will give you nothing more or less than the original image for every entry, no matter what type. It skips a lot of info that isn't necessary for parsing. It also skips mips since it's a lot easier to simply apply a matrix transformation to the original. Storing font info is skipped but, I left the info part of the code as a comment. You would still need to take that info and process it into individual characters.

package parsers.wad
{	
	/**
	* WAD3
	* @version	1 - 062717-17:20Cn
	* @author	OneMadGypsy
	* 
	* Redistribution and use in source and binary forms, with or without
	* modification, are permitted provided that the following conditions are met:
	*
	*   - Redistributions of source code must retain this list of conditions 
	* 	   and the following disclaimer.
	* 
	* THIS SOFTWARE IS PROVIDED BY ONEMADGYPSY "AS IS" AND ANY
	* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
	* WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
	* DISCLAIMED. IN NO EVENT SHALL THE ONEMADGYPSY BE LIABLE FOR
	* ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
	* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
	* SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
	* CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
	* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
	* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
	* DAMAGE.
	*/
	
	import flash.utils.ByteArray;
	import flash.utils.Endian;
	
	import flash.display.BitmapData;
	
	public class WAD3
	{
		private static const MAX_PALETTE_COLORS:int	= 256;
		private static const MAX_NAME_LENGTH:int	= 16;
		private static const QCHAR_WIDTH:int		= 16;
		private static const QNUM_GLYPHS:int		= 256;
		
		public static function parse(filename:String):void
		{
			//get wad bytes from lib and prime for parsing
			var bytes:ByteArray	= Content.wad3file(filename);
			bytes.endian		= Endian.LITTLE_ENDIAN;
			bytes.position		= 0;
			
			//check magic
			if (bytes.readMultiByte(4, "iso-8859-1") != "WAD3") throw new Error("Invalid or unsupported WAD file");
			
			//get directory count and start offset
			var cnt:int = bytes.readInt(); 
			var ofs:int = bytes.readInt(); 
		
			var d:int, n:int, i:int;				//dimensions, entry number, palette index
			var width:int, height:int, alpha:uint;
			var offset:int, type:int, name:String;
			var pixels:ByteArray 		= new ByteArray();
			var indexes:ByteArray		= new ByteArray();
			var palette:Vector.<uint>	= new Vector.<uint>(MAX_PALETTE_COLORS);
			var bmd:BitmapData;
			
			n = 0;
			do
			{
				bytes.position = ofs + (n << 5);			//set pointer
				
				offset		= bytes.readInt();
				bytes.position += 8;					/**@skip sizes*/
				type		= bytes.readByte();
				bytes.position += 3;					/**@skip compression and pad*/
				name		= bytes.readMultiByte(MAX_NAME_LENGTH, "iso-8859-1");
			
				bytes.position = offset;				//set pointer
				
				if(type == 0x40 || type == 0x43)
					bytes.position += MAX_NAME_LENGTH;		/**@skip name*/
				
				//store dimensions
				width	= bytes.readInt();
				height	= bytes.readInt();
				
				//qfont:
				if (type == 0x46)
				{ 	width = 256;
				
					/**@skip all font info*/
					bytes.position += (8 + (QNUM_GLYPHS << 2));	//comment this line if the below is uncommented
					
					/* fontInfo needs to be stored somewhere for reference
					
					var rcount:int = bytes.readInt();
					var rwidth:int = bytes.readInt();
					
					var fontInfo:Vector.<Object> = new Vector.<Object>();
					do { 
						fontInfo.push({
							offset:bytes.readUnsignedShort(),
							width:bytes.readUnsignedShort()
						});
					} while (fontInfo.length <= QNUM_GLYPHS);
					
					*/
				}
				
				d = width * height;
				
				if (type == 0x40 || type == 0x43) bytes.position += 16;			/**@skip name*/
				
				if (indexes.length > 0) indexes.clear();				//reset
				
				//segregate palette indexes for this image 
				bytes.readBytes(indexes, 0, d);
				indexes.position = 0;							//prime indexes pointer
				
				if (type == 0x40 || type == 0x43)
					bytes.position += ((d >> 2) + (d >> 4) + (d >> 6) + 2);		/**@skip mipmaps - +2 is pad*/
				else  bytes.position += 2; 						//pad
				
				i = 0;	//prime
				do {	//get appropriate palette
					if (type == 0x40)
						palette[i] = (0xFF << 2 | i << 16 | i << 8 | i);	//black & white
					else 
					{	
						if (i == (MAX_PALETTE_COLORS - 1))
						{	if (name.charAt(0) == "{") alpha = (0x00 << 24);//transparent
						} else alpha = (0xFF << 24);				//opaque
						
						palette[i] = (	alpha | 
								bytes.readUnsignedByte() << 16	| 
								bytes.readUnsignedByte() << 8	| 
								bytes.readUnsignedByte()
							);						//color
					}
				} while (++i < MAX_PALETTE_COLORS);
				
				if (pixels.length > 0) pixels.clear();			//reset
				
				//write all of the image pixels
				do { pixels.writeUnsignedInt( palette[ indexes.readUnsignedByte() ] ) } while (--d);
				
				pixels.position = 0;					//prime pixels pointer
				
				bmd	= new BitmapData(width, height, true);		//instantiate bitmap data
				bmd.setPixels(bmd.rect, pixels);			//set pixels to bitmap data
				
				Content.set_image(name, bmd);				//save bitmap data in lib
				
			} while (++n < cnt);
		}
	}
}

edit: Actually I can get rid of the entry Object. That's some legacy garbage that doesn't apply to my current method.

edit edit: code updated

[MadGypsy]

I now have it down to 1/12th of the texmex time to load and display the same huge wad. And since I've been saying it wrong this entire time... my parser/loader is 91% faster than texmex.

Forever and ever I have been using setPixels() to create a bitmap. That method uses a ByteArray of uint32 colors to set the BitmapData. I decided to see what kind of performance setVector() would generate and it shaved more than a second off the entire time. setVector() uses a Vector.<uint> of colors to set the BitmapData.

[MadGypsy]

I found an "exploit" in flash where I can read/write directly to memory. In little endian it is roughly twice as fast as using a standard byteArray. The concept is simple. Tell application domain memory to be whatever bytes you want to read/write and then read/write them. Doing it this way is much closer to the alloc way that c languages use and is therefore much faster.

I won't be satisfied til my stuff is so fast it is unbelievable that it is flash.

I have not injected this method into any of my scripts yet but, when I do I expect to practically double their speed. At least the bytes parts, which are a grand majority of the work.