Version history for Cyotek WebCopy
<<Back to software description
Changes for v1.5.0.516 - v1.6.0.543 Beta
- New
- The Basic Authentication dialog now allows the prompting of future passwords to be disabled
- Preview functionality of the Test URI dialog now supports a subset of images
- Added proxy settings to Test URI dialog
- Added new options for specifying custom headers [#219]
- The Test URI dialog now allows the configuration of content encoding, custom headers and URI transforms [#296]
- Added stand-alone version of the test URI tool
- The Capture Form dialog will now try and find the best match if multiple forms are detected on a page [#230]
- Changed
- The layout of the Test URI dialog has been reworked [#296]
- Setup has a new option to determine if icons should be created for stand-alone tools
- Setup has a new option to determine if experimental 64bit versions of tools should be installed
- Minor improvements to External Tools dialog
- Minor start-up improvements
- The option to save headers with the project file is now enabled by default for new projects
- Some context menu items which disappeared from virtualised lists have now been re-instated
- If the character set for a HTML document isn't explicitly specified, WebCopy will now try and autodetect an appropriate value [#303]
- Removed
- Removed the Content tab from the Link Properties dialog
- Removed the unused Modified URI field from the Link Properties dialog
- Removed Find more user agents online link from the Edit User Agents dialog
- Removed the Allow Editing checkbox from the Link Properties dialog
- Removed the Disable Updates flag from link information
- Fixed
- The Basic Authentication dialog truncated long realm text [#312]
- WebCopy no longer tries to unpack custom settings belonging to unloaded extensions [#278]
- WebCopy no longer stores downloaded content against the link information when a 400 or 500 series response is returned
- Output editors in the Test URI dialog now honour the Fixed Font setting
- Project files were no longer being compressed when saved
- Fixed a crash that could occur when running the Empty Meta Data report
- Backup files were not being created when saving projects
- Default external tools configurations were not added when starting WebCopy for the first time
- Some files were still download even if they had been excluded via a rule (regression from 1.4)
- Editing a local file using the build in text editor always used UTF-8 and would corrupt files using a different encoding
- The default user agent was using the file version of the WebCopy client instead of the product version
- The Quick Scan window is now resizeable and remembers its position
- Corrected some settings that weren't being cached
Changes for v1.2.1.348 - v1.2.2.368
- Added
- Added a new optional extension for providing feedback/smiles/frowns or support requests from within the application
- Error diagnosis dialogs now include a reference to the original report
- The Test URI dialog now includes a new tab which lists all links that were detected on the source page
- Fixed
- Errors loading cached RSS feed resulted in the RSS extension from not functioning [0000204]
- Fixed potential exit crash when updating statistics [0000175]
- Fixed a potential issue where the last character in a directory path could be removed [0000213]
- Fixed a crash which could occur when setting file timestamps [000216]
- Fixed a crash that could occur changing the URI of a project containg forms, and the form URI hadn't been set
- Fixed a crash that could occur when right clicking some list views
- CLI tool didn't handle invalid command line arguments as well as it could [0000223]
- The Website Size dialog could crash with a divide by zero exception [0000222]
- The WebCopy CLI now creates missing output directories when requesting to download a single URL into a file [0000224]
Changes for v1.1.3.304 Beta - v1.2.1.348
- Added
- The Origin Report setting now has a new option to embed the original URL as a comment in the body of the HTML
- Changed
- The Download All Resources option is now automatically set for new projects [0000193]
- The Directory Character option is now automatically set to / for new projects
- The Update Local Timestamps is now automatically set for new projects
- Fixed
- A crash no longer occurs opening the Options dialog and the languages folder doesn't exist [0000187]
- A crash no longer occurs opening the Options dialog and duplicate languages are present [0000186]
- A crash could occur when loading the sitemap and shell icons were enabled [0000190]
- Fixed a number of issues that could occur after opening or saving a project and the MRU was updated (0000161, 0000176, 0000177)
- WebCopy no longer aborts the crawl after trying to download a URL with the same name as a file system reserved word [0000195]
- WebCopy wasn't detecting flash movies in an object tag [0000173]
- Data URI's with padded data weren't processed correctly [0000197]
- Fixed trying to run the CLI and specifying a project file that did not exist [0000200]
- Added speculative fix for a crash generating sitemaps - this is a common issue yet we've been completely unable to reproduce it in any of our test scripts or saved projects. If anyone can supply information on how to trigger this crash it would be gratefully received! [0000160]
- Fixed a regression where it was possible for redirects to get stuck in an infinite loop
- Minor improvements to URI exception reporting
- RSS entries would duplicate themselves depending on if the feed was accessed via HTTP or HTTPS. Note that a side effect of this fix will result all entries being marked as unread
- Fixed a possible crash that could occur when trying to load a themed font [0000203]
- Fixed a crash that could occur if a rule had a empty pattern [0000188]
- WebCopy will now try to remove base tags after completing a crawl [0000191]
Changes for v1.1.1.4 - v1.1.3.304 Beta
- Added
- Added a new Keep Alive setting. Setting this to false can help prevent the "The server committed a protocol violation. Section=ResponseStatusLine" crawl failure [0000002]
- Added a new Prefix Mode setting. This setting allows you to force URI's to either have or remove the www prefix, useful for avoiding duplicated files when copying a website which uses a mix of prefixed and non-prefixed URI's
- Added the ability to replace sections of a URI when crawling documents
- Added a new report to view non-HTTP links
- (Experimental) Added new Extract Data URIs setting. Enabling this option will extract inlined images using the data: protocol into separate files.
- Changed
- Setup should now automatically uninstall previous versions
- Numerous changes to how plugins are discovered, loaded and configured. Due to no longer storing plugin details in the Windows Registry, this will cause any disabled plugins to be re-enabled
- WebCopy will now correctly report non-HTTP links such as mailto: or ftp: as skipped rather than silently ignoring them
- Internal engine changes [0000062]
- Removed
- The project scan and repair tool is no longer included in setup
- Fixed
- WebCopy could incorrectly exclude some URL's believing them to be mailto: links
- Fixed several occurrences when a crash could occur when invalid path characters were present in URL segments
- Some HTML tags appeared as "Unknown" in list views
- URI's would be incorrectly combined if the relative URI was just query string and the source URI already had a query string
- Download percentages were calculated incorrectly
- Fixed a crash that could occur after copying a website if the Update local time stamps option was set
- Report viewer didn't show external URI's
- Fixed case-sensitivity issues in some built-in reports
- Fixed a crash that could occur if non-NBT files were present in report folders with the rpt extension
- Fixed a startup crash if the addins folder didn't exist [0000112]
- Fixed a crash that could occur when trying to calculate the depth of a URI [0000116]
- Fixed a crash that occurred if a project with a blank URI was opened and the user then attempted to browse to the blank URI [0000120]
- Fixed an issue where Setup sometimes wouldn't replace files
Changes for v1.1.1.3 - v1.1.1.4
- Fixed
- Fixed an issue where downloaded files would ignore the save folder and start from the root directory if the URL was malformed and included a double slash after the domain, e.g. http://example.com//image1.png
- Fixed a crash that would occur when trying to process a data: URI greater than 65519 characters
- The Capture Form tool was incorrectly using the id attribute of form elements instead of the name
- Setup was incorrectly downloading .NET Framework 4.5.2 setup if .NET Framework 4.6 was installed
- Speculative fix for loading date times from project files
- Speculative fix for odd crashes when opening the Capture Form dialog
- Added
- Double clicking an entry in the Cookies list view of the Test URL dialog now displays the details of the selected item
Changes for 1.1.1.2 Beta - v1.1.1.3
- Fixed
- WebCopy wasn't scanning the contents of style elements correctly
- @import CSS rules were not being remapped if they did not use url() notation
- Fixed a crash which could occur when a request made via the Test URL dialog failed, and no response was available
- Fixed an issue where the Capture Form dialog sometimes did not list forms for a page when it should have
Changes for v1.1.0.2 Beta - v1.1.1.0 Beta
- new Added support for the srcset attribute
- new You can now specify custom attributes to include in link scanning
- new When logging an exception, diagnosis actions are such as new version downloads or links to workarounds are now displayed, if applicable
- new Now supports finding links via the 300 "Multiple Choices" HTTP status code
- new Slight improvements to scan performance
- fix Fixed a crash that occurred if you entered an invalid path into the Save Folder field then attempted to copy a website
- fix Fixed a problem where projects using a sub path and the Crawl above root URI option could save duplicate URI's into the project, causing a crash when attempting to reload the project
- fix Fixed a issue where sitemaps belonging to projects using a sub path and the Crawl above root URI option were corrupt
- fix When changing settings via the main Options dialog, some settings would not be applied as the old versions were cached
- fix Fixed a start up crash that occurred if the externaltools.xml file was present, but invalid
- fix The XPath expression for fix HTML attribute scan rules that used regular expressions to transform only part of value of the attribute were incorrectly merged the transformed value
- fix The link checker tools would not report URI's that weren't found if the URI was also external
- fix The samples default tool link was incorrect
- fix Demo project corrections
Changes for v1.0.10.1 - v1.1.0.2 Beta
- new Added a new option to control whether or not new pre-release (beta) versions are included in update checks
- new 64bit versions of WebCopy (GUI / CLI) and Link Checker (GUI / CLI) are now available
- new You can now choose to display all errors, or only errors detected during the current scan in the Errors tab
- new Activating list items in the different result tabs now opens the appropriate properties dialog
- new Added useragent, prehead and no-prehead command line options to wcopy.exe
- new Uses alpha version of new exception logging library
- removed Disabled glass effects unless using Windows Vista or Windows 7
- fix Build was deploying the .NET 3.5 version of Luminitix
- fix If posting a form failed, the copy was automatically cancelled, but the reason why the post failed was not available
- fix Pressing enter in the sitemap tree view could cause the link properties dialog to be displayed twice
- fix The Link Checker GUI / CLI clients and the WebCopy CLI client no longer require the source URI to be qualified with the scheme, and will automatically add http if no scheme is present
- fix CLI tools now correctly report errors
- fix Default user agents of CLI tools were malformed
- fix In certain circumstances, command line arguments would not be parsed correctly
- fix 401 challenge dialogs were not displaying correctly, instead a "Cross-thread operation not valid" message would be displayed in the log
Changes for v1.0.9.0 - v1.0.10.0
- new Reinstated digital signatures
- new When posting a form, existing values will be automatically merged with the user defined custom values
- new Added a new tool for capturing a form, making it much easier to extract the basic tokens for posting a form
- new Cookies are now supported by the Test URL dialog when making multiple requests from the same domain, including their own tab for viewing
- new All standard HTTP verbs are now supported by the Test URL dialog
- change The Test URL dialog has been split in two, so that the result content is always visible
- change The Rule Editor, Form Editor and Test URL dialogs are now all resizable
- fix Fixed an issue where some form values would not be encoded correctly
- fix GZip and deflate compressed data is now decompressed during the download, rather than after the entire content has been download
- fix The HTML view in the Test URL dialog now correctly updates each time a new request is made
- fix WebCopy would often given file names a numeric suffix even if there was no reason to
- fix If WebCopy tried to shrink a file name to fit within path limits, it incorrectly started by trimming the extension, instead of the name
- fix WebCopy failed to shrink file names where the base path was above 248 characters and promptly crashed
- fix Some files were missing from the setup that prevented exception reports from being submitted (regression from previous version)
- fix Fixed a duplicated shortcut between Rules and Test URI
- fix Exiting WebCopy while the RSS extension was updating caused a crash
- fix Fixed an issue where files could be loaded with the wrong encoding when remapping documents, causing subtle corruption with the final output
- fix The Scan Project repair tool crashed on start up (regression from previous version)
- fix Opening a project always marked it as changed, causing the UI to prompt to save changes unnecessarily
Changes for v1.0.8.0 - v1.0.9.0
- Changes and new features
- Warning Deprecated The prefix with the website url / prefix with the website domain name option of a crawl project has been deprecated and will be removed in a future update.
- Information Experimental Added the ability to specify additional hosts. This allows you to include multiple domains per project, for example a CDN
- Information Experimental Added proxy server support
- Activating an item in either the Request Headers or Response Headers tabs of the Test URI dialog now displays the header information in a dialog for easy viewing/copying
- The contents of the Select Mime Types dialog are now sorted
- Items in the Title Replacements and Forms editors can now be reordered via drag and drop
- Added a helper tool for backing up and restoring settings, or for resetting settings to default values
- Added a stand-alone update check tool
- Bug fixes
- The Status Code column in the Results list is now no longer cleared when an action is performed that didn't involve an HTTP request, such as remapping the local file
- The value of the Play Sounds setting wasn't being honoured by the Crawl Complete dialog
- The prefix with the website url / prefix with the website domain name option of a crawl project now defaults to prefix with the website domain name for new projects
- Pressing enter in the Post Values field of the Test URI dialog no longer activates the default button on the dialog
- Fixed an issue where only the end of a host was inspected when checking if a given URI was a sub domain of another. For example, it would incorrectly return that static.oneexample.com was a subdomain of example.com
- Fixed an issue it was possible WebCopy wouldn't prompt to save changes when exiting
- An error is no longer displayed if you open a project saved using a newer version of WebCopy. The project will now be opened where possible, but a warning will now be displayed
- Repeatedly clicking column headers in sortable lists now correctly cycles between Ascending, Descending and None, instead of only Ascending and Descending.
- Fixed a problem where clicking the Add button in the Form Editor would clone the active form, including the internal ID of the form which should be unique, leading to crashes
- Fixed an issue where settings were both loaded and saved using thread specific culture data, which could cause a crash if the computer culture information was subsequently changed. All settings are now saved and load using an invariant culture.
- A crash no longer occurs if font information cannot be read correctly from stored settings
Changes for v1.0.7.3 - v1.0.7.6
- Changes and new features
- Panels in Option dialogs now load on demand
- Option pages are now only initialized when requested by the appropriate dialog
- Removed status code 520 (origin error) from the list of supported codes for automated error reporting during a crawl
- The Image Viewer window no longer defaults to Fit when displaying an image, but now defaults to Actual size
- Added additional themes for configuring the appearance of the GUI client window
- Bug Fixes
- Fixed an issue cloning LinkInfo objects which hopefully is responsible for a rare Cannot access a disposed object crash using the Quick Scan dialog
- Fixed an issue where the sitemap tree view could be populated up to 3 times rather than the expected once when opening a project
- Dynamic options in the Options dialog are now positioned more sensibly in relation to the options label and editor, and other options in the same group
- Fixed a problem where tool tips did not display under certain conditions, or could display the wrong (or blank) text
- Extension mapping for dropped files was case sensitive
- Reworked tool bar layout code to prevent overflowed buttons
- Removed a number of integration hacks
Changes for v1.0.7.2 - v1.0.7.3
- Added manifest so that when running under Windows 8.1 / Server 2012 R2 the OS version is correctly reported.
Changes for v1.0.7.0 - v1.0.7.2
- Fixed a crash that occurred when building a sitemap for a project that had root level URI's with a query string, but without a document name.
Changes for v1.0.6.1 - v1.0.7.0
- Changes and new features
- Experimental: Added a new option to simplify the sitemap treeview. When this option is set, folder containers are no longer displayed if the folder only has a single page
- Experimental: Modifying a rule now reapplies rules to the sitemap, allowing easier sitemap manipulation without having to rescan the site. Note this feature only works on the current contents of the link map, if the linkmap is incomplete due to existing rules a rescan will be required regardless.
- Sorting of the sitemap now uses natural sorting, so names appear in a logical order, e.g. 1, 2, 10 rather than 1, 10, 2
- The Rule and Forms lists now default to sorted
- List views that support sorted columns now use natural sorting
- When building a sitemap, folders are no longer generated for URI's that match except for differing query strings
- New API to allow plugin authors to add additional functionality to application windows when they are created
- Sitemap treeview now displays URI's relative to the base URI
- Rules that do not use the Use Full Uri flag now also strip out the leading path of the base URI. For example, if the base URI of the project is http://demo.cyotek.com/staticwebsite/ and the current URI being crawled is http://demo.cyotek.com/staticwebsite/blog/page1.html, the text used by the rule engine will be /blog/page1.html
- The Differences tab now lists all URI's which are new to the last scan, in addition to existing checks of modification dates. Due to the introduction of this setting, all URI's will be marked as new for existing projects, until that project is rescanned and saved.
- Removed the Use Modified Uri rule flag
- Bug fixes
- Fixed a problem where clicking OK on the Edit Rule dialog saved changes even if there was a validation error and the user subsequently clicked Cancel
- Fixed a problem where the Quick Scan dialog failed without finding any URL's if the Inclusion / Exclusions options were set
- Fixed a problem where page titles and descriptions containing HTML entities were not decoded
- Fixed a problem where the sitemap could include URI's containing query strings, even if the strip query string segments option was set
- Disabled Glass effects on dialogs when running under terminal services connections
- Fixed a problem where source redirect URI's were not excluded, and appeared in the sitemap
- Outgoing links for an existing link are no longer cleared if the link is excluded for any reason
- Fixed a issue where the skipped status of a URI wasn't reset correctly
Changes for v1.0.5.0 - v1.0.6.1
- This update resolves a number of bugs, including one that can leave a WebCopy project in state where it can be no longer opened. Other bug fixes include problems trying to view the properties of URI's containing spaces, and an issue where a crawl could be cancelled without warning.
- A smattering of new functionality is also available, the Additional URL's section now allows relative URI's, and the Website Size dialog now supports additional display options.
- Some performance improvements have also been made for when updating the user interface after loading very large projects, but there is more to be done.
- Fixed a crash which occurred when opening the Website Size dialog and the linkmap was empty.
- Fixed a potential crash in 1.0.6.0 due to some left behind debug code
Changes for v1.0.4.0 - v1.0.5.0
- Changes and new features
- Added a new Origin Report setting. This setting allows the generation of either single or multiple origin files which are saved alongside downloaded content and include the source URI. This new setting can be found in the Advanced section of a projects properties.
- Request headers are now stored with each URI the same as response headers and will be saved in the project for later retrieval if the Save Headers option is set.
- The Headers tab in the Link Properties dialog now displays request headers
- Added new options for setting the Accept and Accept-Language request headers.
- Removed status code 406 (not acceptable) from the list of supported codes for automated error reporting during a crawl
- Temporary files are now created in the folder where the website is being downloaded to, speeding up the final moving of files after a successful download, and avoiding potential problems if the disk where the temp folders is located doesn't have sufficient space to store the downloaded file.
- Quick Scan dialog now displays progress state while performing the scan
- Bug fixes
- Referring URI's were being incorrectly set since the last update
- Total download size is now incremented correctly even if the content length was reported as zero by the server
- Filtering a grouped list didn't preserve groups when previously filtered items were restored
- URI's that have a status code of 406 (not acceptable) now have the correct skip reason associated with them
- The Content tab of the Test Link dialog didn't always correctly display returned content
- Fixed a crash that could occur when attempting to sort a list
- Fixed a crash if the Content-Type response header contained a space before the encoding name, for example text/css; charset= UTF-8.
- Fixed a crash if the Content-Type response header specified utf8 instead of utf-8.
- Fixed a crash that could occur if the source URI couldn't be decoded correctly
- Deflate encoding now once again correctly works after being broken in a previous build
- Fixed an occasional crash attempting to get the short form URI pattern when creating rules from an existing URI
- Fixed a problem where tool bars didn't wrap correctly if a new tool bar had to be placed on a new row
Changes for v1.0.3.3 - v1.0.4.0
- Changes and new features
- Experimental: Added the basis of a "quick scan" feature. This scans the top level of the website for unique absolute URI's (removing bookmarks and query strings) and is useful for getting a quick overview of the top level structure of the website, making it easier to detect and exclude pages that have no benefit to copy (such as new thread / reply thread pages in a forum). As with other experimental features, this will be expanded over future updates.
- By default, new projects will now remap local file extensions based on their file type if no existing extension is present
- Removed status code 502 (bad gateway), 503 (service unavailable) and 504 (gateway timeout) from the list of supported codes for automated error reporting during a crawl
- Bug fixes
- Fixed a problem where when using the Excluded and Add Rule commands, the generated URI was invalid if there was a mix of www prefixed and non prefixed URI's
- Fixed a crash that occurred when clicking the Test URI button in the Form Editor and the URI of the project is invalid
- Fixed a problem where occasionally it was possible to execute two crawls at once, causing the second crawl to crash
- Fixed a crash that occurred when WebCopy tried to map the folder aspect of a URI and a file already existed with the same name
- Fixed a crash that occurred when submitting the remove missing links dialog for a project without a valid URI
Changes for v1.0.2.2 - v1.0.3.3
- Changes and new features
- Status bar now shows pending crawl requests.
- The progress bar now attempts to show current process based on total requests. It's not hugely accurate as it doesn't take into account the size of each request, but is better than a marquee! Windows 7 and 8 users will see the same behaviour on the taskbar progress.
- Added support for the data attribute of the object tag.
- Removed status code 500 (internal server error) from the list of supported codes for automated error reporting during a crawl
- Removed downloaded file hash calculation as they currently aren't used by WebCopy
- Bug fixes
- Fixed a problem where GZIP compressed content was downloaded incorrectly if the response headers didn't include a content length
- Fixed a problem where some users experienced a startup crash when initializing fonts
- Fixed a build problem that meant some exception reports were missing information
- Fixed a problem where buffers were incorrectly being processed when downloading which could lead to a potential crash or corrupt file if the response header didn't include a content length, and otherwise just did extra repeated work if a length was available
- Fixed a crash that could occur when crawling websites that had many nested branches of links
- Temporary files generated during the analysis of a website are now deleted as soon as they are no longer required, rather than only once the crawl has completed
- The "is missing" check was ignoring HTTP status codes and only going from the scan index