Commit graph

133 commits

Author SHA1 Message Date
wraitii
35ed55cfd6 ParamNode: switch to utf8, ToXML -> ToXMLString, test tweak, PCH tweak
This changes ParamNode to use UTF8 values internally (XMB files are UTF8
since cb9d0733ef).
This removes the need for a lot of conversions, speeding things up and
allows cleaning up the validator interface & a few other callsites.
ConstructJSVal could be a tad slower because of UTF8->16 conversions
within Spidermonkey; but the difference is unlikely to be noticeable in
practica.

Also:
- Changes `ToXML` to `ToXMLString` for clarity.
- Add a simple "op" test & show a particular behaviour of merge nodes
that I intend to change somewhat in D3830.
- Remove Component.h from simulation2 PCH - brought in too much.

Tested by: langbart
Differential Revision: https://code.wildfiregames.com/D3834
This was SVN commit r25228.
2021-04-11 09:23:10 +00:00
wraitii
0e7fafebe1 Refuse to serialize NaN values.
NaN values could not be serialised safely because of the multiple
possible NaN numbers.
Since NaN values are usually the result of bugs or dangerous code, it
seems simpler to refuse to serialise them.

(D3205 was a safe-serialization alternative, should the need arise).

Fixes #1879

Differential Revision: https://code.wildfiregames.com/D3729
This was SVN commit r25151.
2021-03-28 16:48:25 +00:00
wraitii
87a2c3347f Make real directory paths on the VFS predictable (retry)
This is the same commit as 4bb31f084e, reverted in 5d1899785a. Noted
issues were fixed.

Partial revert/fix of 2f19cf86d3 and 2567fee329.

Before this diff, it was possible for a mod containing a cache/ or a
config/ folder to get written to incorrectly.
The issue is VFS can map multiple directories to one 'virtual' path, for
reading mods. However, writing data is problematic: which path to
choose?
The only viable solution is to use a path relative to the highest
priority directory encountered in the VFS path, or write paths could be
'hijacked' by lower-priority mods.

This fixes these issues by:
- Adding a new lookup mode ('Real-path') that explicitly picks the real
path relative to the highest-priority subdirectory in the VFS Path.
- Preventing overwriting a real directory with a lower priority one in
general.
- Revert c0c8132dd4's GetRealPath change, re-introducing the function as
GetOriginalPath.

This also cleans up some duplication that led to empty mod folders in
the user mod path, and cleans up loading the 'user' mod.
It also makes it explicit that a directory must be passed to Mount().

Note that the new 'realpath' lookup can still be somewhat complex with
many mount points at various hierarchy levels, but it is at least
predictable/deterministic without having to be careful about populating
order.

Fixes #2553

Differential Revision: https://code.wildfiregames.com/D3728
This was SVN commit r25107.
2021-03-23 12:46:59 +00:00
wraitii
5d1899785a Revert 4bb31f084e
4bb31f084e led to an issue in windows-specific code not detected by the
tests. I'll debug later.

This was SVN commit r25105.
2021-03-22 19:52:07 +00:00
wraitii
4bb31f084e Make real directory paths on the VFS predictable.
Partial revert/fix of 2f19cf86d3 and 2567fee329.

Before this diff, it was possible for a mod containing a cache/ or a
config/ folder to get written to incorrectly.
The issue is VFS can map multiple directories to one 'virtual' path, for
reading mods. However, writing data is problematic: which path to
choose?
The only viable solution is to use a path relative to the highest
priority directory encountered in the VFS path, or write paths could be
'hijacked' by lower-priority mods.

This fixes these issues by:
- Adding a new lookup mode ('Real-path') that explicitly picks the real
path relative to the highest-priority subdirectory in the VFS Path.
- Preventing overwriting a real directory with a lower priority one in
general.
- Revert c0c8132dd4's GetRealPath change, re-introducing the function as
GetOriginalPath.

This also cleans up some duplication that led to empty mod folders in
the user mod path, and cleans up loading the 'user' mod.

Note that the new 'realpath' lookup can still be somewhat complex with
many mount points at various hierarchy levels, but it is at least
predictable/deterministic without having to be careful about populating
order.

Fixes #2553

Differential Revision: https://code.wildfiregames.com/D3217
This was SVN commit r25104.
2021-03-22 18:49:42 +00:00
vladislavbelov
7a71011149 Refactors smart pointers creation, reduces code duplication and improves exception safety.
This was SVN commit r24915.
2021-02-13 23:53:40 +00:00
wraitii
e2ff38453f Fix lag on OOS with AIs
Following 9fc6c3c897, the OOS debugSerializer would try to serialize AI
objects entirely. This is extremely slow (might be an infinite loop in
some cases).
Instead, it should print the result of Serialize() in those cases.

(The original idea was that it would print more debugging information,
but in practice it seems to mostly print things that show up when
diffing but aren't actually sources of OOS, so essentially garbage).

Fixes #5917

Differential Revision: https://code.wildfiregames.com/D3361
This was SVN commit r24641.
2021-01-16 10:33:00 +00:00
wraitii
7460d0e56e Revert dad2857538 / Keep track of serialized objects using a GCHashTable
This reverts dad2857538. That diff had two issues:
- It modifies the JS objects, which means subsequent serialization in
quicksave are 'dirty'.
- It doesn't work with non-extensible objects. That's rather annoying,
and has already caused problems.
It also revert f0faab7a42, which was necessary because of the second
issue.

Fixes #5908

Differential Revision: https://code.wildfiregames.com/D3336
This was SVN commit r24563.
2021-01-12 18:43:45 +00:00
wraitii
9fc6c3c897 Correctly serialize/deserialize user-defined JS objects.
Generalize component/AI serialization system to any user-defined JS
object. This includes Vector2D/3D, fixing an old issue.
As with components/AI, JS Objects may implement a Serialize/Deserialize
function to store custom data instead of the default, which attemps to
serialize all enumerable properties.

Fixes #4698

Differential Revision: https://code.wildfiregames.com/D2746
This was SVN commit r24462.
2020-12-27 17:18:13 +00:00
wraitii
5b46ce0778 Use templates to replace explicit serialization helpers.
By using templates appripriately we can remove the need for explicit
specification of serializers, making it easier to serialize container
types and to write new serialization helpers.

Direct serialization calls haven't been replaced in this diff.

Comments by: vladislavbelov
Differential Revision: https://code.wildfiregames.com/D3207
This was SVN commit r24427.
2020-12-19 09:10:37 +00:00
wraitii
48c31ce03f Update random_shuffle usage for C++17
C++14 deprecates and C++17 removes functions we use for random sounds.
This makes FastRand() fit the URG interface, and uses the appropriate
function.

Refs #5862

Differential Revision: https://code.wildfiregames.com/D3107
This was SVN commit r24302.
2020-11-30 15:52:16 +00:00
wraitii
fd8f5abd2e [SM52 2/2] Update to Spidermonkey 52 APIs.
No particularly noteworthy changes, as most complex API changes were
already supported in SM45 and done.
The addition of JSStructuredCloneData allows to remove our custom class.

Changes:
- InformalValueTypeName is back in the API, so remove our
implementation.
- Stop using JSRuntime entirely in favour of JSContext*
- JSPropertyDescriptor is renamed.
- CompartmentOptions are tweaked slightly (no functional changes)
- JS::Construct - API update.
- JSClass split - API update.
- A js.msg error message was removed, so we had to use a different one.
- Tests fix: fix comparison of union instances
- Disable warning in spidermonkey Vector.h
- Update error reporting to SM52 (minor API updates)
- Ignore warnings about unused return values (would come from OOM, which
isn't recoverable)

Most of the patching was done by Itms.

Tested by: Stan, Freagarach
Fixes #4893

Differential Revision: https://code.wildfiregames.com/D3095
This was SVN commit r24203.
2020-11-18 14:39:04 +00:00
wraitii
25490bfec3 Improve JS Exception handling.
- Check for pending exceptions after function calls and script
executions.
- Call LOGERROR instead of JS_ReportError when there is a conversion
error in FromJSVal, since that can only be called from C++ (where JS
errors don't really make sense). Instead, C++ callers of FromJSVal
should handle the failure and, themselves, either report an error or
simply do something else.
- Wrap JS_ReportError since that makes updating it later easier.

This isn't a systematical fix since ToJSVal also ought return a boolean
for failures, and we probably should trigger errors instead of warnings
on 'implicit' conversions, rather a preparation diff.

Part of the SM52 migration, stage: SM45 compatible (actually SM52
incompatible, too).

Based on a patch by: Itms
Comments by: Vladislavbelov, Stan`
Refs #742, #4893

Differential Revision: https://code.wildfiregames.com/D3093
This was SVN commit r24187.
2020-11-15 18:29:17 +00:00
wraitii
aa15066c69 Rename ScriptRuntime to ScriptContext
SM52 essentially replaces JSRuntime with JSContext (though JSContext
itself was replaced with JSCompartment).
To prepare for this migration, rename all Runtime-related things to
Context.

Part of the SM52 migration, stage: SM45 compatible.

Patch by: Itms
Refs #4893

Differential Revision: https://code.wildfiregames.com/D3091
This was SVN commit r24181.
2020-11-14 10:57:50 +00:00
wraitii
ee0d204bf6 Wrap JSAutoRequest and replace usage of JSContext* with the wrapper.
JSAutoRequest is required before calling into most JSAPI methods, for GC
reasons.
Calling it is required and fragile as one must not forget.
Further, SM52 and later make manipulating JSContext* dangerous as that
can cross Compartment(Realm in SM68) barriers (and ScriptInterface now
matches a Compartment).

The solution to both problems is to avoid using JSContext* in 0 A.D.
itself. To achieve this, a Request class is introduced, and must be used
to access a JSContext* from a scriptInterface. Further, Request is
passed to other ScriptInterface functions isntead of JSContext*, making
it obvious that the caller has already called it, reducing errors and
redundant JSAutoRequest calls.
Only JSNative functions now get a naked JSContext* without protection,
but the likelihood of forgetting a request is lower since many
ScriptInterface functions now expect it.

JSContext* is directly passed to JSAPI functions only.

Part of the SM52 migration, stage: SM45 compatible

Based on a patch by: Itms
Tested By: Freagarach
Refs #4893

Differential Revision: https://code.wildfiregames.com/D3088
This was SVN commit r24176.
2020-11-13 13:18:22 +00:00
wraitii
dad2857538 Use Symbols to store JS object references when serialising and delete ObjectIDCache
When serialising JS objects, we keep track of any encountered object,
and serialize it only once. Any further serialisation instead stores an
ID referring to the original object (essentially an opaque pointer).
The trouble of course is to have a unique, persistent identifier for
such an object.
svn uses an ObjectIDCache, essentially a "JS Object -> ID" map (which
internally is essentially a "JS heap pointer -> ID" map).

JS, since ES15, includes a "Symbol" primitive type, which is a unique,
immutable identifier. They are also not iterable by for..in or
GetOwnPropertyName or related.
This means they can be used to store the tag directly on the object
(since it's impossible overwrite a user property).
Thanks to this, we can forgo ObjectIDCache in the serializers, and since
following D2897 it becomes unused, we can delete it, along with the
Finalization code it used.


Part of SM52 migration, stage: SM45-compatible changes.

Patch by: Itms
Tested By: Freagarach
Refs #4893

Differential Revision: https://code.wildfiregames.com/D3085
This was SVN commit r24167.
2020-11-12 06:40:19 +00:00
wraitii
939002f0dc RangeManager: Grid for 2D array, enum cleanups.
Range manager has several `std::vector` for fixed-size arrays and 2D
grids. By using proper data structures, the code readability is
improved.
This also moves around the LosVisibility enum.

Comments by: Stan`, nani
Differential Revision: https://code.wildfiregames.com/D2770
This was SVN commit r23769.
2020-06-13 09:05:40 +00:00
Stan
ebd9786268 Add missing cpp files in last commit refs D2503
This was SVN commit r23704.
2020-05-28 18:18:00 +00:00
vladislavbelov
e2f0e409f0 Adds containers serialization tests, std::set serialization and movement semantics
Reviewed By: elexis
Tested By: elexis, Freagarach
Differential Revision: https://code.wildfiregames.com/D2352
This was SVN commit r23044.
2019-10-04 21:30:52 +00:00
Itms
6230bbea24 Update of the scripting tests. Test some modding features and update JS::Value integer limits.
Reviewed By: wraitii
Differential Revision: https://code.wildfiregames.com/D1850
This was SVN commit r22513.
2019-07-19 15:41:27 +00:00
elexis
8a32b0b3d4 Fix some gcc 8 and gcc 9 compiler warnings that were thrown 4500 times.
Refs #5294
Differential Revision: https://code.wildfiregames.com/D2055
Reviewed By: Vladislav
This was SVN commit r22443.
2019-07-09 00:18:48 +00:00
wraitii
c1027f478c Test we can destroy INVALID_ENTITY
This is part of the spec and wasn't unit-tested for.

Differential Revision: https://code.wildfiregames.com/D1735
This was SVN commit r22036.
2019-01-06 16:32:50 +00:00
wraitii
24812b06d7 Revert b56f7f39d4 as it triggers an assertion failure.
This was SVN commit r22032.
2019-01-05 22:38:00 +00:00
wraitii
b56f7f39d4 Change internals of EntityMap to use an std::vector
This generally makes this code cleaner and easier to maintain, while
also being more modern and fully tested.

Looked at by: leper
Differential Revision: https://code.wildfiregames.com/D8
This was SVN commit r22029.
2019-01-05 18:19:59 +00:00
wraitii
af39d45973 Add tests for the Entitymap helper.
In preparation for D8.

Differential Revision: https://code.wildfiregames.com/D1722
This was SVN commit r22027.
2019-01-05 16:37:09 +00:00
elexis
89e339dd16 Remove VFS cache, because it is less effective and less efficient than the OS cache (and partially redundant with higher level application caches).
Patch By: Sandarac
Discussed with: Philip, echotangoecho, Bezerra
Fixes #4072.
Differential Revision: https://code.wildfiregames.com/D587
This was SVN commit r20639.
2017-12-10 17:33:03 +00:00
fatherbushido
ad7d7ec1e9 Fix a disabled template manager test.
Fix #4522
Patch by: Sandarac.
Reviewed by: elexis.
Differential Revision: https://code.wildfiregames.com/D968
This was SVN commit r20326.
2017-10-22 15:39:50 +00:00
leper
ce580f0de0 Always delete CMapReader. Patch by Sandarac. Fixes #4154.
This fixes an assertion failure in ScriptEngine that can occur when
closing
the game while in the loading screen.

Reviewed By: vladislavbelov, leper
Differential Revision: https://code.wildfiregames.com/D684
This was SVN commit r20035.
2017-08-25 00:37:48 +00:00
fatherbushido
de9e76cbf1 Add a test to the component manager to show a crash in dynamic subscriptions when components unsubscribe during deletion. Reviewed by wraitii.
Differential Revision: https://code.wildfiregames.com/D346
This was SVN commit r19423.
2017-04-17 07:55:44 +00:00
elexis
d093f714d7 Use XML files instead of hardcoded C++ code in the template manager to universally change template properties at load time.
Patch By: leper
Differential Revision: https://code.wildfiregames.com/D215
Fixes #2951

This was SVN commit r19302.
2017-03-16 19:56:12 +00:00
leper
3a85d0b3fa Remove unused parameter from template loading code
Templates should be the same for all players, techs can be used to
provide
different lists of templates if that is needed.

Reviewed By: Itms
Differential Revision: https://code.wildfiregames.com/D25
This was SVN commit r19142.
2017-01-15 18:41:53 +00:00
Itms
490a3a27f6 Fix the cache directory appearing during tests.
Summary: Refs #4419.

Test Plan: When running tests, make sure nothing is created.

Reviewers: leper

Reviewed By: leper
Subscribers: leper, Vulcan

Differential Revision: https://code.wildfiregames.com/D1
This was SVN commit r19062.
2016-12-25 22:03:30 +00:00
Itms
a27dc31b0e Remove some obsolete backref logic from the deserializer. It was introduced in earlier versions of the SpiderMonkey API.
Name some variables more logically in one test.

Fixes #4244

This was SVN commit r18993.
2016-11-24 14:47:49 +00:00
elexis
b18f74da44 Remove trailing whitespace and whitespace in empty lines of source/ except source/third_party/.
This was SVN commit r18987.
2016-11-23 11:18:37 +00:00
elexis
3008533c8e Add a template manager test that reveals an inheritance error in ParamNode.cpp which can trigger an OOS on rejoin. Patch by leper, refs #4316.
This was SVN commit r18941.
2016-11-15 13:51:29 +00:00
Itms
9b794593db SpiderMonkey 38 upgrade: 32/35
JS_GetStringCharsAndLength was removed, use
JS_Get{Latin1,TwoByte}StringCharsAndLength instead.

Actually handle strings in both the Latin1 and TwoByte cases since we
need to.
This saves some space when serializing and also when running as they are
stored that way in the vm.
Also handle the error case.

Patch by leper.
For more information:
https://blog.mozilla.org/javascript/2014/07/21/slimmer-and-faster-javascript-strings-in-firefox/
Addresses https://bugzilla.mozilla.org/show_bug.cgi?id=1037869

This was SVN commit r18686.
2016-09-02 16:51:09 +00:00
elexis
6eaf76d653 Actually seed the random number generator used in the simulation. Reviewed by sanderd17, fixes #4127.
This was SVN commit r18604.
2016-08-14 16:28:54 +00:00
leper
f4898c18d8 SpiderMonkey 38 removes the JS_LookupProperty{,ById} API. Refs #3708.
Use JS_GetProperty{,ById} instead.
Ensure that we break if someone tries to serialize a getter by using
something similar to what we used pre 47a03c3397.
https://bugzilla.mozilla.org/show_bug.cgi?id=1094176

This was SVN commit r17633.
2016-01-11 20:03:33 +00:00
leper
3fd4d07ca1 Test \0 in object property names for serializing.
This was SVN commit r17539.
2015-12-22 19:01:18 +00:00
leper
efda359acc Use new when calling the builtin typed array constructor.
The current code will be forbidden in ES6.

This was SVN commit r17504.
2015-12-19 01:10:13 +00:00
historic_bruno
c0b4e78801 Fixes stream serialization bug on OS X 10.8 and older, which caused instant OOS in multiplayer games, fixes #3108.
Fixes test failures on OS X 10.7 and older, refs #3109

This was SVN commit r17133.
2015-10-15 03:31:30 +00:00
leper
2aef62d65f Handle backrefs properly for maps and sets. Fixes #3374.
Allocate the tag for the backreference before deserializing the content,
to match the order of serializing.

This was SVN commit r16959.
2015-08-30 04:51:16 +00:00
leper
c9893b3dd0 Test serialization of nested maps. Refs #3164, #3374.
This was SVN commit r16954.
2015-08-30 00:06:39 +00:00
leper
b8a20c958c Do not serialize properties for Sets. Serialization tests for Map and Set.
This was SVN commit r16620.
2015-05-04 01:41:09 +00:00
Ykkrosh
bb9de80dd4 Convert CConsole to take UTF-8 strings.
This avoids vswprintf failures when printing non-ASCII char* strings
from CLogger into the console.

Also convert ScriptInterface::ToString to return UTF-8, to avoid some
utf8_from_wstring calls.

Also remove some unused and redundant CConsole functions.

This was SVN commit r16333.
2015-02-14 01:49:34 +00:00
Ykkrosh
e06a7b37d8 Convert debug_printf to take UTF-8 strings instead of wchar_t.
This fixes the problem where passing a non-ASCII string to
debug_printf(L"%hs", s) caused vswprintf_s to fail on Linux (because it
doesn't know what encoding the char* is meant to have). Now debug
messages will remain as UTF-8 until they reach the OS.

Fixes #3021.

This was SVN commit r16332.
2015-02-14 01:45:13 +00:00
Yves
c02a7e1a7b SpiderMonkey 31 upgrade
This upgrade also introduces exact stack rooting (see to the wiki:
JSRootingGuide) and fixes problems with moving GC. This allows us to
enable generational garbage collection (GGC).
Measurements a few months ago have shown a performance improvement of a
non-visual replay of around 13.5%. This probably varies quite a bit, but
it should be somewhere between 5-20%. Memory usage has also been
improved. Check the forum thread for details.

Thanks to everyone from the team who helped with this directly or
indirectly (review, finding and fixing issues, the required C++11
upgrade, the new autobuilder etc.)! Also thanks to the SpiderMonkey
developers who helped on the #jsapi channel or elsewhere!

Fixes #2462, #2415, #2428, #2684, #1374
Refs #2973, #2669

This was SVN commit r16214.
2015-01-24 14:46:52 +00:00
Ykkrosh
dcf5a2667f CLogger: Use cppformat instead of sys_vswprintf.
sys_vswprintf relies on platform-specific printf implementations, which
vary widely between platforms (in handling of truncation, return values,
use of %s/%S/%hs/%ls for mixing char and wchar_t strings, etc) and are
therefore a pain.

Use cppformat's fmt::sprintf instead, which has very similar syntax to
sprintf but is more C++ish and is portable.

Also, wchar_t is stupid, so use char* strings (which are expected to be
UTF-8) in CLogger. This creates a bit of a pain with changing all
callers to convert to char* strings, but that's their fault for not
using UTF-8 already.

Refs #3011.

This was SVN commit r16182.
2015-01-22 20:30:05 +00:00
Itms
fd93e4ae74 Remove testing of some special templates. We need to find a good way to generate template subsets, refs #2951.
This fixes the tests fail introduced in b05879e151.

This was SVN commit r16033.
2014-12-09 23:20:31 +00:00
leper
c5c394fd17 Fix some disabled tests.
This was SVN commit r15673.
2014-08-24 15:29:53 +00:00