Commit graph

48 commits

Author SHA1 Message Date
Ykkrosh
0f4928bd0a Use dummy IO tracer, to avoid overhead of storing data that will never be used. (See #1143.)
This was SVN commit r11007.
2012-02-02 23:12:36 +00:00
janwas
301854702c add support for removing individual files from VFS and a mechanism for automatically doing so (add filename.extension.DELETED to the same directory/archive), as discussed in meeting.
refactor: split up Invalidate into RemoveFile+RepopulateDirectory
GameSetup: fix: ensure the first mod always overrides game files,
irrespective of timestamp.

This was SVN commit r10610.
2011-11-26 18:37:53 +00:00
Ykkrosh
7064565ff6 # Allow saving and loading single-player games.
Support writing zip files based on in-memory data.
Fix menu sliding animations to be framerate independent.

This was SVN commit r10454.
2011-10-30 00:07:28 +00:00
Ykkrosh
05d7742ec8 Increase trace cache size to cope slightly better with serialization test mode
This was SVN commit r10447.
2011-10-28 13:16:30 +00:00
janwas
34186dd017 refactor file interface. requires workspace update
- separate file_system_util into vfs functions (-> vfs/vfs_util) and
file_system (avoids ugly fs_util namespace prefix)
- get rid of non-portable O_BINARY_NP etc. flags
- use standard O_WRONLY etc. flags instead of LIO_WRITE; but avoid the
need for specifying O_CREAT|O_TRUNC
- only open files for aio when O_DIRECT is specified (which 0ad does
not) - avoids wasting time and security issues
- return file descriptor directly instead of via output param
- waio: safer FCB mechanism that avoids mixing descriptors between lowio
and aio

This was SVN commit r9550.
2011-05-25 10:39:13 +00:00
janwas
5c76bc12fa refactor status code definitions: allow modules to define an array of them (more convenient+efficient).
merge ERR::NO_SYS, NOT_IMPLEMENTED into NOT_SUPPORTED
renderer: remove unnecessary ogl_shader include

This was SVN commit r9447.
2011-05-05 13:03:34 +00:00
janwas
dcec9c4ca9 gracefully handle the case where the file cache is full of data still referenced elsewhere.
fixes #832

This was SVN commit r9437.
2011-05-04 22:19:38 +00:00
janwas
cccd6849a7 cleanup (requires update-workspaces)
lib_errors.cpp: replace with status.cpp, adapt to needs at work
wutil: fix runtime warning reported via feedback box
config: merge CONFIG_PARANOIA and !CONFIG_FINAL into
CONFIG_ENABLE_CHECKS
add openmp, pointer_typedefs.h

This was SVN commit r9410.
2011-05-03 12:38:42 +00:00
janwas
1636f062a3 make VFS thread-safe, per long-standing request by Philip
This was SVN commit r9369.
2011-04-30 15:57:43 +00:00
janwas
2374caac3e major refactor of file/io and alignment code. requires update-workspaces
. completely rewrite waio - use IOCP, add several hardcore
optimizations. now outperforms the
  AS SSD and ATTO benchmarks when writing
. refactor file interface - use LIO_READ instead of 'r', allow access to
file descriptor.
. completely rewrite the IO wrapper. now much more simple, less CPU
overhead, adds
  support for pre-issue/post-completion hooks and preallocation.
  io::Run defaults to simple synchronous IO; use io::Parameters to get
asynchronous.
. add alignment.h with constants and Align() function template (more
efficient than
  round_up for compile-time constants)
. add UniqueRange - similar to C++0x unique_ptr (emulated for C++03),
plus a
  built-in size. avoids expensive thread-safe reference counting in
shared_ptr.

cleanup:
- move fat_time functions into archive_zip
- remove no longer needed io_align and block_cache
- reduce dependencies in sysdep/compiler (move parts to
code_annotation.h)
- move IOCP into separate file (reused by waio)

This was SVN commit r9350.
2011-04-29 19:10:34 +00:00
janwas
dcd192cb60 refactor path interface:
- use wrapper class instead of std::wstring (reduces mixing of
strings/paths; allows safe+easy join via operator/ and convenient
case-insensitive comparison via operator==, avoids NativePathFromString,
similar to boost::filesystem)
- NativePath -> OsPath
- add hash and To/FromJSVal for Path
- add TS_ASSERT_PATH_EQUALS
- replace _wfopen_s with sys_OpenFile
- remove obsolete SortFiles/Directories

This was SVN commit r9107.
2011-03-23 13:36:20 +00:00
janwas
6d25329412 # towards locale-independent pathnames on Linux
c.f.
http://www.wildfiregames.com/forum/index.php?showtopic=14541&st=0&p=217250&#entry217250
and 2011-03-19 meeting

This was SVN commit r9090.
2011-03-21 17:53:13 +00:00
janwas
c9cc7d9784 skip populating directories when mounting and scanning archives => reduces startup time by about half a second when archives are present.
also disable archive stats (currently unused) and refactor directory
creation/association logic.

This was SVN commit r8321.
2010-10-10 10:22:11 +00:00
Ykkrosh
ad02ab726d Expose VFS file priorities.
This was SVN commit r8096.
2010-09-10 20:29:38 +00:00
janwas
c0c8132dd4 fix VFS real path retrieval for files mounted from multiple directories (e.g. mods)
closes #564
GetRealPath was too simplistic and just asked the directory *most
recently mounted* into the file's VFS directory for its path. the
correct solution is to ask the file's loader for its path. the archiver
IFileLoader new returns the archive's pathname as the real path.
also simplified VfsFile's interface.

This was SVN commit r8082.
2010-09-05 21:41:45 +00:00
janwas
9864da20fc vfs: disable error dialog if file not found
SoundGroup.cpp: add proper error reporting if snd_open fails

This was SVN commit r7493.
2010-05-01 18:36:24 +00:00
janwas
d1c5119694 fix: mustn't create directories during mount's vfs path traversal (because they may not be associated with a directory, especially in the common case of mounting into the root directory)
fixes #459

This was SVN commit r7378.
2010-03-20 19:26:12 +00:00
janwas
4059826759 when mounting a mod, don't attempt to create the directories (it's reasonable to assume the mod must exist, there is no value in creating a dummy mod directory - unlike the screenshots/logs etc. mounts)
this fixes write errors because mods/internal doesn't exist in public
SVN checkouts and the default mount logic tries to mount internal anyway
(and now just fails that operation silently)
fixes #458

This was SVN commit r7375.
2010-03-20 18:12:48 +00:00
janwas
1861448e6c needed to split lib into separate headers and source files, which requires lots of drudgery to specify paths for each include filename (no longer relying on same-directory lookups)
also rename wchar -> utf8 to avoid conflict with <wchar.h> (requires
rebuild of workspace)
(unfortunately copying history fails to "502 bad gateway"; had to delete
old + add new independently)

This was SVN commit r7340.
2010-03-01 14:52:58 +00:00
janwas
c2fd939898 license change to the more permissible Expat/MIT license for lib/ (see forum thread)
This was SVN commit r7316.
2010-02-08 16:23:39 +00:00
Ykkrosh
babab3124f Add ERROR_ASSOCIATE for reporting VFS errors
This was SVN commit r7251.
2010-01-06 20:18:07 +00:00
janwas
6ec9128e78 refactor: move reload functionality into ps/Filesystem to allow reloading non-resource files (e.g. scripts); VFS is now responsible for returning VirtualPath and invalidating the changed file/directory
This was SVN commit r7171.
2009-11-06 11:50:04 +00:00
janwas
25717ef768 add VirtualPath API to VFS in preparation for moving hotload logic out of library code and into the game. also rename GetRealPath -> RealPath + some cleanup/documentation
This was SVN commit r7169.
2009-11-06 01:30:16 +00:00
janwas
8fefedb5ae fix: native and vfs directory paths should end with '/' (more code cares about this now)
refactor: simplify dir_watch_Poll interface (returns vector)
wdir_watch: major refactor: use shared_ptr for reference-counting shared
watch state
remove no longer needed path_util functions

This was SVN commit r7168.
2009-11-05 20:46:15 +00:00
janwas
4c44602ea0 add support for hotloading (i.e. reacting to directory change notifications and reloading the affected files)
also clean up VFS a bit.
ogl_tex now tolerates deleting/renaming texture files that are in use
(except for a warning message which can be 'continue'd)

This was SVN commit r7166.
2009-11-04 22:35:54 +00:00
janwas
b48e877f5b VFS fixes+refactoring
- fix: archive_zip: avoid crash if zip file contains empty filenames
- real_directory: enable dir watch
- refactor vfs_tree interface (move recursive calls into VFS)
- VFS: return text representation instead of only printing it

This was SVN commit r7165.
2009-11-04 17:26:54 +00:00
janwas
8a52113e60 huge cleanup and conversion of most string handling (especially paths) to unicode
please note: format strings must be %hs for char* arguments and %ls for
wchar_t*

This was SVN commit r7161.
2009-11-03 21:46:35 +00:00
janwas
5b302658a3 # fix permissions of created directories
also took the opportunity to move file_system_posix to file_system since
a further FS abstraction layer = YAGNI. also namespaced
file_system_util.

This was SVN commit r7074.
2009-08-04 19:57:53 +00:00
janwas
ebac85ee81 # hopefully support non-admin accounts and unwritable directories by moving output folders to home/appdata
the old behavior (using directories under binaries/) can be kept by
passing -writableRoot on the command line.
the first game load will be slower than usual due to re-creation of
cached XMBs.

This was SVN commit r7065.
2009-08-02 11:07:42 +00:00
janwas
c8cfd8b40b # groundwork for separated data/cache etc. directories (XDG)
remove path.cpp (it was based on the premise that all data files lie
below binaries/data)

This was SVN commit r7063.
2009-08-01 19:37:38 +00:00
janwas
9842b8df98 32/64-bit warning drudgery
also ensure off_t is always 64-bit (required at work)

This was SVN commit r6858.
2009-05-05 20:24:06 +00:00
janwas
c4ae6001f9 fix relics of GPL comment insertion (removing empty description comments and in some cases adding new file descriptions)
test_XeroXMB.h: fix warning

This was SVN commit r6851.
2009-05-03 11:47:38 +00:00
Ykkrosh
1743645fd3 Simplify the overly-complex CppDoc-breaking file headers
This was SVN commit r6832.
2009-04-18 17:51:05 +00:00
Ykkrosh
214b0e6683 Removed outdated licensing information
This was SVN commit r6831.
2009-04-18 17:17:34 +00:00
Ykkrosh
c9fa7f13d9 Add GPL header
This was SVN commit r6830.
2009-04-18 17:00:33 +00:00
Ykkrosh
7a4dd7b473 Set svn:eol-style=native
This was SVN commit r6828.
2009-04-18 16:14:48 +00:00
janwas
c4654fd8fa bits: avoid sign conversion warning
debug: cleanup
ogl_tex, tex: use full-width types to avoid truncation warnings
wdbg_sym.cpp: skip __suppress and __profile symbols; add additional hex
display for floats; cleanup
wdll_ver.cpp: cppdoc+cleanup (remove non-reentrant C interface, replace
with std::string)
sysdep: add cppdoc

everything else: flag parameters are now consistently size_t instead of
int (avoids warnings, allows slightly better code on x64)

This was SVN commit r6392.
2008-09-18 11:27:55 +00:00
janwas
a6499ff319 fix: ensure directory is populated despite multiple Attach operations
refactor: move populate logic to separate file; VfsDirectory now only
provides the realDirectory storage, while Attach() responsibility is
moved to vfs_populate.

This was SVN commit r6333.
2008-09-06 21:09:58 +00:00
janwas
8e86d29301 dehydra fixes (mostly copy-ctor warnings)
This was SVN commit r6239.
2008-07-17 14:23:51 +00:00
janwas
04127c7af3 fixes/improvements to lib code from work
- topology.cpp: modify interface due to thread-safety issue. caller is
now responsible for ensuring the first _Detect call isn't reentered;
everything else is safe.
- fix thread-safety issue in wnuma; use winit mechanism to ensure it's
ready before use
- VFS now takes a cacheSize parameter (required for being able to
disable read-only file caches for the image loader at work)
- allow dynarray that isn't actually holding memory
- debug_stl: VC9 fix (disable this code except on the exact STL version
on which it was tested)
- zlib, lib_api: changes to macro used to toggle between static and
dynamic linking
- add boost filesystem header in external_libraries
- amd64: cpu_ topology functions are now provided by x86_x64
- cpu: remove cpu_ClockFrequency (dangerous, may be tempting to use
during WHRT init which would cause a crash). use x86_x64_ClockFrequency
or os_cpu_ClockFrequency instead.
- werrno: cope with newer boost versions
- wmman: follow SUSv3 in rejecting zero-length mappings

This was SVN commit r5954.
2008-05-13 19:43:02 +00:00
janwas
c0ed950657 had to remove uint and ulong from lib/types.h due to conflict with other library.
this snowballed into a massive search+destroy of the hodgepodge of
mostly equivalent types we had in use (int, uint, unsigned, unsigned
int, i32, u32, ulong, uintN).

it is more efficient to use 64-bit types in 64-bit mode, so the
preferred default is size_t (for anything remotely resembling a size or
index). tile coordinates are ssize_t to allow more efficient conversion
to/from floating point. flags are int because we almost never need more
than 15 distinct bits, bit test/set is not slower and int is fastest to
type. finally, some data that is pretty much directly passed to OpenGL
is now typed accordingly.

after several hours, the code now requires fewer casts and less
guesswork.

other changes:
- unit and player IDs now have an "invalid id" constant in the
respective class to avoid casting and -1
- fix some endian/64-bit bugs in the map (un)packing. added a
convenience function to write/read a size_t.
- ia32: change CPUID interface to allow passing in ecx (required for
cache topology detection, which I need at work). remove some unneeded
functions from asm, replace with intrinsics where possible.

This was SVN commit r5942.
2008-05-11 18:48:32 +00:00
janwas
cc243f67eb fixes/improvements made at work:
- fix lots of 64-bit warnings
- round_up/down now templates
- avoid warning with definition of byte swap functions; remove
duplication of that in wsdl
- codec_zlib.cpp: avoid domination warning
- vfs/file_cache: VFS is now responsible for handling zero-length files
(no longer considered an error; just returns zero pointer and size=0)
- cpu: export all functions (thus obviating cpu_memcpy_thunk). this
required renaming asm functions and adding thunk functions that call
them
- winit: fix segment merge statement, reinstate /include (otherwise init
functions are stripped by linker)
- wstartup: VC CRT init section definitions have changed in amd64 build;
match their definition
- wsysdep.cpp: more descriptive text for osError = 0

This was SVN commit r5899.
2008-05-01 15:41:42 +00:00
janwas
aebf8cbf68 fix VFS bug preventing directories from being created:
- vfs_lookup: currentPath was of wrong type; simplified and fix create
logic
- archive_zip: fix: correctly handle filenames with /
- vfs: remove workaround for above bug
- io: temporarily disable block caching (works around problem with zip
file created by 7z)

add and use new wdbg_heap module (WIP)
remove mmgr

This was SVN commit r5571.
2008-01-20 16:53:09 +00:00
janwas
a859562ea7 improvements and fixes:
- properly differentiate between buffer/offset alignment and length
alignment (relevant since block size has been increased to 256k)
- use VfsPath for most game paths instead of CStr
- clean up timer interface and implementation
- self-tests no longer crash
- file_cache.cpp: fix for the case where allocation fails (prevent
deleter from seeing a null pointer)
- allocators: move all shared_ptr-related stuff to its own component;
add DummySharedPtr
- codec: disable checksums (important for performance at work)
- File: made into an interface class to avoid export problems. not
entirely sure about this..
- vfs_path.h, path.h, os_path.h: proper fix for using
fs::change_extension and similar utility functions with derivatives of
basic_path
- lib_api: automatically link against import lib if building lib/ as a
DLL
- path_util: remove unused functions (this component is deprecated)
- compiler.h: add INLINE
- Xeromyces.cpp: pass PIVFS so that GetXMBPath works in self-test
(should do this mostly everywhere rather than have one singleton g_VFS)

This was SVN commit r5537.
2008-01-07 20:03:19 +00:00
janwas
8667ea74c8 fixes and improvements
- directoryPosix: replace most methods with boost filesystem (but not
all: the latter cannot efficiently enumerate files AND query their
size/mtime)
- AllocatorChecker: better name for member functions
- file: move the File class here
- trace: bugfix
- io: move UnalignedWriter to write_buffer.cpp (basically the same
thing)
- vfs: remove unnecessary "vfs" warts from variable names
- vfs_tree: VfsFile now stores single Name/Size/MTime fields instead of
the FileInfo record (less clunky)
- vfs_path: use boost filesystem's version of the basename/extension
functions
- lf_alloc: remove (no longer necessary, won't be finished - not worth
the trouble)
- path_util: remove path_foreach_component (replaced by better path
traversal logic) and PathPackage (obsoleted by fs::path)

! resource loading code now receives VfsPath as its filename. there is
also OsPath (native absolute path) and Path (relative to binaries/data)

- tex is now independent of file loading code; it just en/decodes
in-memory buffers
- wdll_ver: clean up, use smart pointer to simplify bailout code
- wsdl: remove nonexistent failure path from calc_gamma (cruised by here
because SDL_SetGamme is failing once after a cold boot at work)
- wsnd: simplify OpenAL DLL search, use boost::filesystem
- wutil: Wow64 redirection is now packaged in a (RAII) class

This was SVN commit r5525.
2007-12-22 18:15:52 +00:00
janwas
63086f4e26 part3: now functional file loading code (not entirely finalized)
(too many changes to list..)

moved tex_* files to lib/tex
archive builder and hotloading modules are disabled.\

This was SVN commit r5518.
2007-12-20 20:14:21 +00:00
janwas
75ab906315 WIP file code, getting pretty close to functional
archive: ArchiveEntry now a mememto and not exposed directly
typedefs for boost::shared_ptr
fixed up dir_util to new interface
add "FileProvider" that does Load/Store and indicates Precedence

vfs: structure has changed. one RealDirectory per OS dir; vfs_tree
stores it; populate uses it to Add() stuff to vfs_tree; lookup does path
traversal and calls populate

This was SVN commit r5499.
2007-12-01 18:32:43 +00:00
janwas
317f98a6c0 WIP of new file code
archive: pretty much done except for trace and archive_builder
IO: io_manager still needs to be fixed/revised (especially WRT buffer
and alignment management)
posix: done
vfs: need to complete vfs_mount and tie it in to vfs.cpp
dir_util: needs further revisions (VFS interface changed since then)

This was SVN commit r5475.
2007-11-20 18:51:07 +00:00