Merge pull request #2 from sqlmapproject/master

update

.gitattributes

@@ -1,5 +1,8 @@
-*.py text eol=lf
 *.conf text eol=lf
+*.md text eol=lf
+*.md5 text eol=lf
+*.py text eol=lf
+*.xml text eol=lf
 
 *_ binary
 *.dll binary

.gitignore

@@ -2,4 +2,5 @@
 output/
 .sqlmap_history
 traffic.txt
-*~
+*~
+.idea/

.travis.yml

@@ -0,0 +1,6 @@
+language: python
+python:
+    - "2.6"
+    - "2.7"
+script:
+    - python -c "import sqlmap; import sqlmapapi"

ISSUE_TEMPLATE.md

@@ -0,0 +1,26 @@
+## What's the problem (or question)?
+<!--- If describing a bug, tell us what happens instead of the expected behavior -->
+<!--- If suggesting a change/improvement, explain the difference from current behavior -->
+
+## Do you have an idea for a solution?
+<!--- Not obligatory, but suggest a fix/reason for the bug, -->
+<!--- or ideas how to implement the addition or change -->
+
+## How can we reproduce the issue?
+<!--- Provide unambiguous set of steps to reproduce this bug. Include command to reproduce, if relevant (you can mask the sensitive data) -->
+1.
+2.
+3.
+4.
+
+## What are the running context details?
+<!--- Include as many relevant details about the running context you experienced the bug/problem in -->
+* Installation method (e.g. `pip`, `apt-get`, `git clone` or `zip`/`tar.gz`):
+* Client OS (e.g. `Microsoft Windows 10`)
+* Program version (`python sqlmap.py --version` or `sqlmap --version` depending on installation): 
+* Target DBMS (e.g. `Microsoft SQL Server`): 
+* Detected WAF/IDS/IPS protection (e.g. `ModSecurity` or `unknown`):
+* SQLi techniques found by sqlmap (e.g. `error-based` and `boolean-based blind`):
+* Results of manual target assessment (e.g. found that the payload `query=test' AND 4113 IN ((SELECT 'foobar'))-- qKLV` works):
+* Relevant console output (if any):
+* Exception traceback (if any):

README.md

@@ -1,6 +1,6 @@
-sqlmap
+# sqlmap
-==
 
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers. It comes with a powerful detection engine, many niche features for the ultimate penetration tester and a broad range of switches lasting from database fingerprinting, over data fetching from the database, to accessing the underlying file system and executing commands on the operating system via out-of-band connections.
 
@@ -18,7 +18,7 @@ You can download the latest tarball by clicking [here](https://github.com/sqlmap
 
 Preferably, you can download sqlmap by cloning the [Git](https://github.com/sqlmapproject/sqlmap) repository:
 
-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
 
 sqlmap works out of the box with [Python](http://www.python.org/download/) version **2.6.x** and **2.7.x** on any platform.
 
@@ -33,8 +33,8 @@ To get a list of all options and switches use:
 
     python sqlmap.py -hh
 
-You can find a sample run [here](https://gist.github.com/stamparm/5335217).
+You can find a sample run [here](https://asciinema.org/a/46601).
-To get an overview of sqlmap capabilities, list of supported features and description of all options and switches, along with examples, you are advised to consult the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki).
+To get an overview of sqlmap capabilities, list of supported features and description of all options and switches, along with examples, you are advised to consult the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
 
 Links
 ----
@@ -45,9 +45,6 @@ Links
 * Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
 * User's manual: https://github.com/sqlmapproject/sqlmap/wiki
 * Frequently Asked Questions (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
-* Mailing list subscription: https://lists.sourceforge.net/lists/listinfo/sqlmap-users
-* Mailing list RSS feed: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap
-* Mailing list archive: http://news.gmane.org/gmane.comp.security.sqlmap
 * Twitter: [@sqlmap](https://twitter.com/sqlmap)
 * Demos: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
 * Screenshots: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots
@@ -55,8 +52,14 @@ Links
 Translations
 ----
 
+* [Bulgarian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-bg-BG.md)
 * [Chinese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-zh-CN.md)
 * [Croatian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-hr-HR.md)
+* [French](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-fr-FR.md)
 * [Greek](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-gr-GR.md)
 * [Indonesian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-id-ID.md)
+* [Italian](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-it-IT.md)
+* [Japanese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-ja-JP.md)
 * [Portuguese](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-pt-BR.md)
+* [Spanish](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-es-MX.md)
+* [Turkish](https://github.com/sqlmapproject/sqlmap/blob/master/doc/translations/README-tr-TR.md)

doc/CHANGELOG.md

@@ -1,14 +1,12 @@
-# Version 1.0 (upcoming)
+# Version 1.0 (2016-02-27)
 
 * Implemented support for automatic decoding of page content through detected charset.
 * Implemented mechanism for proper data dumping on DBMSes not supporting `LIMIT/OFFSET` like mechanism(s) (e.g. Microsoft SQL Server, Sybase, etc.).
 * Major improvements to program stabilization based on user reports.
 * Added new tampering scripts avoiding popular WAF/IPS/IDS mechanisms.
-* Added support for setting Tor proxy type together with port.
 * Fixed major bug with DNS leaking in Tor mode.
 * Added wordlist compilation made of the most popular cracking dictionaries.
-* Added support for mnemonics substantially helping user with program setup.
+* Implemented multi-processor hash cracking routine(s).
-* Implemented multi-processor hash cracking routine(s) on Linux OS.
 * Implemented advanced detection techniques for inband and time-based injections by usage of standard deviation method.
 * Old resume files are now deprecated and replaced by faster SQLite based session mechanism.
 * Substantial code optimization and smaller memory footprint.
@@ -28,9 +26,72 @@
 * Added switch `--check-waf` for checking of existence of WAF/IPS/IDS protection.
 * Added switch `--schema` to enumerate DBMS schema: shows all columns of all databases' tables.
 * Added switch `--count` to count the number of entries for a specific table or all database(s) tables.
-* Major improvements to switches --tables and --columns.
+* Major improvements to switches `--tables` and `--columns`.
-* Takeover switch --os-pwn improved: stealthier, faster and AV-proof.
+* Takeover switch `--os-pwn` improved: stealthier, faster and AV-proof.
-* Added switch --mobile to imitate a mobile device through HTTP User-Agent header.
+* Added switch `--mobile` to imitate a mobile device through HTTP User-Agent header.
+* Added switch `-a` to enumerate all DBMS data.
+* Added option `--alert` to run host OS command(s) when SQL injection is found.
+* Added option `--answers` to set user answers to asked questions during sqlmap run.
+* Added option `--auth-file` to set HTTP authentication PEM cert/private key file.
+* Added option `--charset` to force character encoding used during data retrieval.
+* Added switch `--check-tor` to force checking of proper usage of Tor.
+* Added option `--code` to set HTTP code to match when query is evaluated to True.
+* Added option `--cookie-del` to set character to be used while splitting cookie values.
+* Added option `--crawl` to set the crawling depth for the website starting from the target URL.
+* Added option `--crawl-exclude` for setting regular expression for excluding pages from crawling (e.g. `"logout"`).
+* Added option `--csrf-token` to set the parameter name that is holding the anti-CSRF token.
+* Added option `--csrf-url` for setting the URL address for extracting the anti-CSRF token.
+* Added option `--csv-del` for setting the delimiting character that will be used in CSV output (default `,`).
+* Added option `--dbms-cred` to set the DBMS authentication credentials (user:password).
+* Added switch `--dependencies` for turning on the checking of missing (non-core) sqlmap dependencies.
+* Added switch `--disable-coloring` to disable console output coloring.
+* Added option `--dns-domain` to set the domain name for usage in DNS exfiltration attack(s).
+* Added option `--dump-format` to set the format of dumped data (`CSV` (default), `HTML` or `SQLITE`).
+* Added option `--eval` for setting the Python code that will be evaluated before the request.
+* Added switch `--force-ssl` to force usage of SSL/HTTPS.
+* Added switch `--hex` to force usage of DBMS hex function(s) for data retrieval.
+* Added option `-H` to set extra HTTP header (e.g. `"X-Forwarded-For: 127.0.0.1"`).
+* Added switch `-hh` for showing advanced help message.
+* Added option `--host` to set the HTTP Host header value.
+* Added switch `--hostname` to turn on retrieval of DBMS server hostname.
+* Added switch `--hpp` to turn on the usage of HTTP parameter pollution WAF bypass method.
+* Added switch `--identify-waf` for turning on the thorough testing of WAF/IPS/IDS protection.
+* Added switch `--ignore-401` to ignore HTTP Error Code 401 (Unauthorized).
+* Added switch `--invalid-bignum` for usage of big numbers while invalidating values.
+* Added switch `--invalid-logical` for usage of logical operations while invalidating values.
+* Added switch `--invalid-string` for usage of random strings while invalidating values.
+* Added option `--load-cookies` to set the file containing cookies in Netscape/wget format.
+* Added option `-m` to set the textual file holding multiple targets for scanning purposes.
+* Added option `--method` to force usage of provided HTTP method (e.g. `PUT`).
+* Added switch `--no-cast` for turning off payload casting mechanism.
+* Added switch `--no-escape` for turning off string escaping mechanism.
+* Added option `--not-string` for setting string to be matched when query is evaluated to False.
+* Added switch `--offline` to force work in offline mode (i.e. only use session data).
+* Added option `--output-dir` to set custom output directory path.
+* Added option `--param-del` to set character used for splitting parameter values.
+* Added option `--pivot-column` to set column name that will be used while dumping tables by usage of pivot(ing).
+* Added option `--proxy-file` to set file holding proxy list.
+* Added switch `--purge-output` to turn on safe removal of all content(s) from output directory.
+* Added option `--randomize` to set parameter name(s) that will be randomly changed during sqlmap run.
+* Added option `--safe-post` to set POST data for sending to safe URL.
+* Added option `--safe-req` for loading HTTP request from a file that will be used during sending to safe URL.
+* Added option `--skip` to skip testing of given parameter(s).
+* Added switch `--skip-static` to skip testing parameters that not appear to be dynamic.
+* Added switch `--skip-urlencode` to skip URL encoding of payload data.
+* Added switch `--skip-waf` to skip heuristic detection of WAF/IPS/IDS protection.
+* Added switch `--smart` to conduct thorough tests only if positive heuristic(s).
+* Added option `--sql-file` for setting file(s) holding SQL statements to be executed (in case of stacked SQLi).
+* Added switch `--sqlmap-shell` to turn on interactive sqlmap shell prompt.
+* Added option `--test-filter` for test filtration by payloads and/or titles (e.g. `ROW`).
+* Added option `--test-skip` for skipping tests by payloads and/or titles (e.g. `BENCHMARK`).
+* Added switch `--titles` to turn on comparison of pages based only on their titles.
+* Added option `--tor-port` to explicitly set Tor proxy port.
+* Added option `--tor-type` to set Tor proxy type (`HTTP` (default), `SOCKS4` or `SOCKS5`).
+* Added option `--union-from` to set table to be used in `FROM` part of UNION query SQL injection.
+* Added option `--where` to set `WHERE` condition to be used during the table dumping.
+* Added option `-X` to exclude DBMS database table column(s) from enumeration.
+* Added option `-x` to set URL of sitemap(.xml) for target(s) parsing.
+* Added option `-z` for usage of short mnemonics (e.g. `"flu,bat,ban,tec=EU"`).
 
 # Version 0.9 (2011-04-10)
 
@@ -43,7 +104,7 @@
 * Extended old `--dump -C` functionality to be able to search for specific database(s), table(s) and column(s), option `--search`.
 * Added support to tamper injection data with option `--tamper`.
 * Added automatic recognition of password hashes format and support to crack them with a dictionary-based attack.
-* Added support to enumerate roles on Oracle, --roles switch.
+* Added support to enumerate roles on Oracle, `--roles` switch.
 * Added support for SOAP based web services requests.
 * Added support to fetch unicode data.
 * Added support to use persistent HTTP(s) connection for speed improvement, switch `--keep-alive`.
@@ -88,18 +149,18 @@
 * Major bugs fixed.
 * Cleanup of UDF source code repository, https://svn.sqlmap.org/sqlmap/trunk/sqlmap/extra/udfhack.
 * Major code cleanup.
-* Added simple file encryption/compression utility, extra/cloak/cloak.py, used by sqlmap to decrypt on the fly Churrasco, UPX executable and web shells consequently reducing drastically the number of anti-virus softwares that mistakenly mark sqlmap as a malware.
+* Added simple file encryption/compression utility, extra/cloak/cloak.py, used by sqlmap to decrypt on the fly Churrasco, UPX executable and web shells consequently reducing drastically the number of anti-virus software that mistakenly mark sqlmap as a malware.
 * Updated user's manual.
 * Created several demo videos, hosted on YouTube (http://www.youtube.com/user/inquisb) and linked from http://sqlmap.org/demo.html.
 
 # Version 0.8 release candidate (2009-09-21)
 
-* Major enhancement to the Microsoft SQL Server stored procedure heap-based buffer overflow exploit (--os-bof) to automatically bypass DEP memory protection.
+* Major enhancement to the Microsoft SQL Server stored procedure heap-based buffer overflow exploit (`--os-bof`) to automatically bypass DEP memory protection.
 * Added support for MySQL and PostgreSQL to execute Metasploit shellcode via UDF 'sys_bineval' (in-memory, anti-forensics technique) as an option instead of uploading the standalone payload stager executable.
 * Added options for MySQL, PostgreSQL and Microsoft SQL Server to read/add/delete Windows registry keys.
 * Added options for MySQL and PostgreSQL to inject custom user-defined functions.
-* Added support for --first and --last so the user now has even more granularity in what to enumerate in the query output.
+* Added support for `--first` and `--last` so the user now has even more granularity in what to enumerate in the query output.
-* Minor enhancement to save the session by default in 'output/hostname/session' file if -s option is not specified.
+* Minor enhancement to save the session by default in 'output/hostname/session' file if `-s` option is not specified.
 * Minor improvement to automatically remove sqlmap created temporary files from the DBMS underlying file system.
 * Minor bugs fixed.
 * Major code refactoring.
@@ -108,13 +169,13 @@
 
 * Adapted Metasploit wrapping functions to work with latest 3.3 development version too.
 * Adjusted code to make sqlmap 0.7 to work again on Mac OSX too.
-* Reset takeover OOB features (if any of --os-pwn, --os-smbrelay or --os-bof is selected) when running under Windows because msfconsole and msfcli are not supported on the native Windows Ruby interpreter. This make sqlmap 0.7 to work again on Windows too.
+* Reset takeover OOB features (if any of `--os-pwn`, `--os-smbrelay` or `--os-bof` is selected) when running under Windows because msfconsole and msfcli are not supported on the native Windows Ruby interpreter. This make sqlmap 0.7 to work again on Windows too.
 * Minor improvement so that sqlmap tests also all parameters with no value (eg. par=).
 * HTTPS requests over HTTP proxy now work on either Python 2.4, 2.5 and 2.6+.
 * Major bug fix to sql-query/sql-shell features.
-* Major bug fix in --read-file option.
+* Major bug fix in `--read-file` option.
 * Major silent bug fix to multi-threading functionality.
-* Fixed the web backdoor functionality (for MySQL) when (usually) stacked queries are not supported and --os-shell is provided.
+* Fixed the web backdoor functionality (for MySQL) when (usually) stacked queries are not supported and `--os-shell` is provided.
 * Fixed MySQL 'comment injection' version fingerprint.
 * Fixed basic Microsoft SQL Server 2000 fingerprint.
 * Many minor bug fixes and code refactoring.
@@ -136,32 +197,32 @@
 * Major enhancement to make the comparison algorithm work properly also on url not stables automatically by using the difflib Sequence Matcher object;
 * Major enhancement to support SQL data definition statements, SQL data manipulation statements, etc from user in SQL query and SQL shell if stacked queries are supported by the web application technology;
 * Major speed increase in DBMS basic fingerprint;
-* Minor enhancement to support an option (--is-dba) to show if the current user is a database management system administrator;
+* Minor enhancement to support an option (`--is-dba`) to show if the current user is a database management system administrator;
-* Minor enhancement to support an option (--union-tech) to specify the technique to use to detect the number of columns used in the web application SELECT statement: NULL bruteforcing (default) or ORDER BY clause bruteforcing;
+* Minor enhancement to support an option (`--union-tech`) to specify the technique to use to detect the number of columns used in the web application SELECT statement: NULL bruteforcing (default) or ORDER BY clause bruteforcing;
-* Added internal support to forge CASE statements, used only by --is-dba query at the moment;
+* Added internal support to forge CASE statements, used only by `--is-dba` query at the moment;
-* Minor layout adjustment to the --update output;
+* Minor layout adjustment to the `--update` output;
 * Increased default timeout to 30 seconds;
 * Major bug fix to correctly handle custom SQL "limited" queries on Microsoft SQL Server and Oracle;
 * Major bug fix to avoid tracebacks when multiple targets are specified and one of them is not reachable;
 * Minor bug fix to make the Partial UNION query SQL injection technique work properly also on Oracle and Microsoft SQL Server;
-* Minor bug fix to make the --postfix work even if --prefix is not provided;
+* Minor bug fix to make the `--postfix` work even if `--prefix` is not provided;
 * Updated documentation.
 
 # Version 0.6.3 (2008-12-18)
 
 * Major enhancement to get list of targets to test from Burp proxy (http://portswigger.net/suite/) requests log file path or WebScarab proxy (http://www.owasp.org/index.php/Category:OWASP_WebScarab_Project) 'conversations/' folder path by providing option -l <filepath>;
 * Major enhancement to support Partial UNION query SQL injection technique too;
-* Major enhancement to test if the web application technology supports stacked queries (multiple statements) by providing option --stacked-test which will be then used someday also by takeover functionality;
+* Major enhancement to test if the web application technology supports stacked queries (multiple statements) by providing option `--stacked-test` which will be then used someday also by takeover functionality;
-* Major enhancement to test if the injectable parameter is affected by a time based blind SQL injection technique by providing option --time-test;
+* Major enhancement to test if the injectable parameter is affected by a time based blind SQL injection technique by providing option `--time-test`;
 * Minor enhancement to fingerprint the web server operating system and the web application technology by parsing some HTTP response headers;
 * Minor enhancement to fingerprint the back-end DBMS operating system by parsing the DBMS banner value when -b option is provided;
-* Minor enhancement to be able to specify the number of seconds before timeout the connection by providing option --timeout #, default is set to 10 seconds and must be 3 or higher;
+* Minor enhancement to be able to specify the number of seconds before timeout the connection by providing option `--timeout #`, default is set to 10 seconds and must be 3 or higher;
-* Minor enhancement to be able to specify the number of seconds to wait between each HTTP request by providing option --delay #;
+* Minor enhancement to be able to specify the number of seconds to wait between each HTTP request by providing option `--delay #`;
-* Minor enhancement to be able to get the injection payload --prefix and --postfix from user;
+* Minor enhancement to be able to get the injection payload `--prefix` and `--postfix` from user;
 * Minor enhancement to be able to enumerate table columns and dump table entries, also when the database name is not provided, by using the current database on MySQL and Microsoft SQL Server, the 'public' scheme on PostgreSQL and the 'USERS' TABLESPACE_NAME on Oracle;
-* Minor enhancemet to support also --regexp, --excl-str and --excl-reg options rather than only --string when comparing HTTP responses page content;
+* Minor enhancemet to support also `--regexp`, `--excl-str` and `--excl-reg` options rather than only `--string` when comparing HTTP responses page content;
-* Minor enhancement to be able to specify extra HTTP headers by providing option --headers. By default Accept, Accept-Language and Accept-Charset headers are set;
+* Minor enhancement to be able to specify extra HTTP headers by providing option `--headers`. By default Accept, Accept-Language and Accept-Charset headers are set;
-* Minor improvement to be able to provide CU (as current user) as user value (-U) when enumerating users privileges or users passwords;
+* Minor improvement to be able to provide CU (as current user) as user value (`-U`) when enumerating users privileges or users passwords;
 * Minor improvements to sqlmap Debian package files;
 * Minor improvement to use Python psyco (http://psyco.sourceforge.net/) library if available to speed up the sqlmap algorithmic operations;
 * Minor improvement to retry the HTTP request up to three times in case an exception is raised during the connection to the target url;
@@ -175,10 +236,10 @@
 
 # Version 0.6.2 (2008-11-02)
 
-* Major bug fix to correctly dump tables entries when --stop is not specified;
+* Major bug fix to correctly dump tables entries when `--stop` is not specified;
 * Major bug fix so that the users' privileges enumeration now works properly also on both MySQL < 5.0 and MySQL >= 5.0;
 * Major bug fix when the request is POST to also send the GET parameters if any have been provided;
-* Major bug fix to correctly update sqlmap to the latest stable release with command line --update;
+* Major bug fix to correctly update sqlmap to the latest stable release with command line `--update`;
 * Major bug fix so that when the expected value of a query (count variable) is an integer and, for some reasons, its resumed value from the session file is a string or a binary file, the query is executed again and its new output saved to the session file;
 * Minor bug fix in MySQL comment injection fingerprint technique;
 * Minor improvement to correctly enumerate tables, columns and dump tables entries on Oracle and on PostgreSQL when the database name is not 'public' schema or a system database;
@@ -191,20 +252,20 @@
 * Major bug fix to blind SQL injection bisection algorithm to handle an exception;
 * Added a Metasploit Framework 3 auxiliary module to run sqlmap;
 * Implemented possibility to test for and inject also on LIKE statements;
-* Implemented --start and --stop options to set the first and the last table entry to dump;
+* Implemented `--start` and `--stop` options to set the first and the last table entry to dump;
-* Added non-interactive/batch-mode (--batch) option to make it easy to wrap sqlmap in Metasploit and any other tool;
+* Added non-interactive/batch-mode (`--batch`) option to make it easy to wrap sqlmap in Metasploit and any other tool;
 * Minor enhancement to save also the length of query output in the session file when retrieving the query output length for ETA or for resume purposes;
 * Changed the order sqlmap dump table entries from column by column to row by row. Now it also dumps entries as they are stored in the tables, not forcing the entries' order alphabetically anymore;
-* Minor bug fix to correctly handle parameters' value with % character.
+* Minor bug fix to correctly handle parameters' value with `%` character.
 
 # Version 0.6 (2008-09-01)
 
 * Complete code refactor and many bugs fixed;
 * Added multithreading support to set the maximum number of concurrent HTTP requests;
-* Implemented SQL shell (--sql-shell) functionality and fixed SQL query (--sql-query, before called -e) to be able to run whatever SELECT statement and get its output in both inband and blind SQL injection attack;
+* Implemented SQL shell (`--sql-shell`) functionality and fixed SQL query (`--sql-query`, before called `-e`) to be able to run whatever SELECT statement and get its output in both inband and blind SQL injection attack;
-* Added an option (--privileges) to retrieve DBMS users privileges, it also notifies if the user is a DBMS administrator;
+* Added an option (`--privileges`) to retrieve DBMS users privileges, it also notifies if the user is a DBMS administrator;
-* Added support (-c) to read options from configuration file, an example of valid INI file is sqlmap.conf and support (--save) to save command line options on a configuration file;
+* Added support (`-c`) to read options from configuration file, an example of valid INI file is sqlmap.conf and support (`--save`) to save command line options on a configuration file;
-* Created a function that updates the whole sqlmap to the latest stable version available by running sqlmap with --update option;
+* Created a function that updates the whole sqlmap to the latest stable version available by running sqlmap with `--update` option;
 * Created sqlmap .deb (Debian, Ubuntu, etc.) and .rpm (Fedora, etc.) installation binary packages;
 * Created sqlmap .exe (Windows) portable executable;
 * Save a lot of more information to the session file, useful when resuming injection on the same target to not loose time on identifying injection, UNION fields and back-end DBMS twice or more times;
@@ -216,8 +277,8 @@
 * Improved XML files structure;
 * Implemented the possibility to change the HTTP Referer header;
 * Added support to resume from session file also when running with inband SQL injection attack;
-* Added an option (--os-shell) to execute operating system commands if the back-end DBMS is MySQL, the web server has the PHP engine active and permits write access on a directory within the document root;
+* Added an option (`--os-shell`) to execute operating system commands if the back-end DBMS is MySQL, the web server has the PHP engine active and permits write access on a directory within the document root;
-* Added a check to assure that the provided string to match (--string) is within the page content;
+* Added a check to assure that the provided string to match (`--string`) is within the page content;
 * Fixed various queries in XML file;
 * Added LIMIT, ORDER BY and COUNT queries to the XML file and adapted the library to parse it;
 * Fixed password fetching function, mainly for Microsoft SQL Server and reviewed the password hashes parsing function;
@@ -225,7 +286,7 @@
 * Enhanced logging system: added three more levels of verbosity to show also HTTP sent and received traffic;
 * Enhancement to handle Set-Cookie from target url and automatically re-establish the Session when it expires;
 * Added support to inject also on Set-Cookie parameters;
-* Implemented TAB completion and command history on both --sql-shell and --os-shell;
+* Implemented TAB completion and command history on both `--sql-shell` and `--os-shell`;
 * Renamed some command line options;
 * Added a conversion library;
 * Added code schema and reminders for future developments;
@@ -237,19 +298,19 @@
 # Version 0.5 (2007-11-04)
 
 * Added support for Oracle database management system
-* Extended inband SQL injection functionality (--union-use) to all other possible queries since it only worked with -e and --file on all DMBS plugins;
+* Extended inband SQL injection functionality (`--union-use`) to all other possible queries since it only worked with `-e` and `--file` on all DMBS plugins;
 * Added support to extract database users password hash on Microsoft SQL Server;
 * Added a fuzzer function with the aim to parse HTML page looking for standard database error messages consequently improving database fingerprinting;
 * Added support for SQL injection on HTTP Cookie and User-Agent headers;
-* Reviewed HTTP request library (lib/request.py) to support the extended inband SQL injection functionality. Splitted getValue() into getInband() and getBlind();
+* Reviewed HTTP request library (lib/request.py) to support the extended inband SQL injection functionality. Split getValue() into getInband() and getBlind();
 * Major enhancements in common library and added checkForBrackets() method to check if the bracket(s) are needed to perform a UNION query SQL injection attack;
-* Implemented --dump-all functionality to dump entire DBMS data from all databases tables;
+* Implemented `--dump-all` functionality to dump entire DBMS data from all databases tables;
-* Added support to exclude DBMS system databases' when enumeration tables and dumping their entries (--exclude-sysdbs);
+* Added support to exclude DBMS system databases' when enumeration tables and dumping their entries (`--exclude-sysdbs`);
 * Implemented in Dump.dbTableValues() method the CSV file dumped data automatic saving in csv/ folder by default;
 * Added DB2, Informix and Sybase DBMS error messages and minor improvements in xml/errors.xml;
 * Major improvement in all three DBMS plugins so now sqlmap does not get entire databases' tables structure when all of database/table/ column are specified to be dumped;
 * Important fixes in lib/option.py to make sqlmap properly work also with python 2.5 and handle the CSV dump files creation work also under Windows operating system, function __setCSVDir() and fixed also in lib/dump.py;
-* Minor enhancement in lib/injection.py to randomize the number requested to test the presence of a SQL injection affected parameter and implemented the possibilities to break (q) the for cycle when using the google dork option (-g);
+* Minor enhancement in lib/injection.py to randomize the number requested to test the presence of a SQL injection affected parameter and implemented the possibilities to break (q) the for cycle when using the google dork option (`-g`);
 * Minor fix in lib/request.py to properly encode the url to request in case the "fixed" part of the url has blank spaces;
 * More minor layout enhancements in some libraries;
 * Renamed DMBS plugins;
@@ -260,21 +321,21 @@
 
 * Added DBMS fingerprint based also upon HTML error messages parsing defined in lib/parser.py which reads an XML file defining default error messages for each supported DBMS;
 * Added Microsoft SQL Server extensive DBMS fingerprint checks based upon accurate '@@version' parsing matching on an XML file to get also the exact patching level of the DBMS;
-* Added support for query ETA (Estimated Time of Arrival) real time calculation (--eta);
+* Added support for query ETA (Estimated Time of Arrival) real time calculation (`--eta`);
-* Added support to extract database management system users password hash on MySQL and PostgreSQL (--passwords);
+* Added support to extract database management system users password hash on MySQL and PostgreSQL (`--passwords`);
 * Added docstrings to all functions, classes and methods, consequently released the sqlmap development documentation <http://sqlmap.org/dev/>;
-* Implemented Google dorking feature (-g) to take advantage of Google results affected by SQL injection to perform other command line argument on their DBMS;
+* Implemented Google dorking feature (`-g`) to take advantage of Google results affected by SQL injection to perform other command line argument on their DBMS;
 * Improved logging functionality: passed from banal 'print' to Python native logging library;
-* Added support for more than one parameter in '-p' command line option;
+* Added support for more than one parameter in `-p` command line option;
-* Added support for HTTP Basic and Digest authentication methods (--basic-auth and --digest-auth);
+* Added support for HTTP Basic and Digest authentication methods (`--basic-auth` and `--digest-auth`);
-* Added the command line option '--remote-dbms' to manually specify the remote DBMS;
+* Added the command line option `--remote-dbms` to manually specify the remote DBMS;
-* Major improvements in union.UnionCheck() and union.UnionUse() functions to make it possible to exploit inband SQL injection also with database comment characters ('--' and '#') in UNION query statements;
+* Major improvements in union.UnionCheck() and union.UnionUse() functions to make it possible to exploit inband SQL injection also with database comment characters (`--` and `#`) in UNION query statements;
-* Added the possibility to save the output into a file while performing the queries (-o OUTPUTFILE) so it is possible to stop and resume the same query output retrieving in a second time (--resume);
+* Added the possibility to save the output into a file while performing the queries (`-o OUTPUTFILE`) so it is possible to stop and resume the same query output retrieving in a second time (`--resume`);
-* Added support to specify the database table column to enumerate (-C COL);
+* Added support to specify the database table column to enumerate (`-C COL`);
-* Added inband SQL injection (UNION query) support (--union-use);
+* Added inband SQL injection (UNION query) support (`--union-use`);
 * Complete code refactoring, a lot of minor and some major fixes in libraries, many minor improvements;
 * Reviewed the directory tree structure;
-* Splitted lib/common.py: inband injection functionalities now are moved to lib/union.py;
+* Split lib/common.py: inband injection functionalities now are moved to lib/union.py;
 * Updated documentation files.
 
 # Version 0.3 (2007-01-20)
@@ -282,10 +343,10 @@
 * Added module for MS SQL Server;
 * Strongly improved MySQL dbms active fingerprint and added MySQL comment injection check;
 * Added PostgreSQL dbms active fingerprint;
-* Added support for string match (--string);
+* Added support for string match (`--string`);
-* Added support for UNION check (--union-check);
+* Added support for UNION check (`--union-check`);
 * Removed duplicated code, delegated most of features to the engine in common.py and option.py;
-* Added support for --data command line argument to pass the string for POST requests;
+* Added support for `--data` command line argument to pass the string for POST requests;
 * Added encodeParams() method to encode url parameters before making http request;
 * Many bug fixes;
 * Rewritten documentation files;

doc/CONTRIBUTING.md

@@ -1,38 +1,37 @@
-# Contributing to sqlmap
+# Contributing to sqlmap
-
+
-## Reporting bugs
+## Reporting bugs
-
+
-**Bug reports are welcome**!
+**Bug reports are welcome**!
-Please report all bugs on the [issue tracker](https://github.com/sqlmapproject/sqlmap/issues).
+Please report all bugs on the [issue tracker](https://github.com/sqlmapproject/sqlmap/issues).
-
+
-### Guidelines
+### Guidelines
-
+
-* Before you submit a bug report, search both [open](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aopen+is%3Aissue) and [closed](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aissue+is%3Aclosed) issues to make sure the issue has not come up before. Also, check the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki) for anything relevant.
+* Before you submit a bug report, search both [open](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aopen+is%3Aissue) and [closed](https://github.com/sqlmapproject/sqlmap/issues?q=is%3Aissue+is%3Aclosed) issues to make sure the issue has not come up before. Also, check the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki) for anything relevant.
-* Make sure you can reproduce the bug with the latest development version of sqlmap.
+* Make sure you can reproduce the bug with the latest development version of sqlmap.
-* Your report should give detailed instructions on how to reproduce the problem. If sqlmap raises an unhandled exception, the entire traceback is needed. Details of the unexpected behaviour are welcome too. A small test case (just a few lines) is ideal.
+* Your report should give detailed instructions on how to reproduce the problem. If sqlmap raises an unhandled exception, the entire traceback is needed. Details of the unexpected behaviour are welcome too. A small test case (just a few lines) is ideal.
-* If you are making an enhancement request, lay out the rationale for the feature you are requesting. *Why would this feature be useful?*
+* If you are making an enhancement request, lay out the rationale for the feature you are requesting. *Why would this feature be useful?*
-* If you are not sure whether something is a bug, or want to discuss a potential new feature before putting in an enhancement request, the [mailing list](https://lists.sourceforge.net/lists/listinfo/sqlmap-users) is a good place to bring it up.
+
-
+## Submitting code changes
-## Submitting code changes
+
-
+All code contributions are greatly appreciated. First off, clone the [Git repository](https://github.com/sqlmapproject/sqlmap), read the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki) carefully, go through the code yourself and [drop us an email](mailto:dev@sqlmap.org) if you are having a hard time grasping its structure and meaning. We apologize for not commenting the code enough - you could take a chance to read it through and [improve it](https://github.com/sqlmapproject/sqlmap/issues/37).
-All code contributions are greatly appreciated. First off, clone the [Git repository](https://github.com/sqlmapproject/sqlmap), read the [user's manual](https://github.com/sqlmapproject/sqlmap/wiki) carefully, go through the code yourself and [drop us an email](mailto:dev@sqlmap.org) if you are having a hard time grasping its structure and meaning. We apologize for not commenting the code enough - you could take a chance to read it through and [improve it](https://github.com/sqlmapproject/sqlmap/issues/37).
+
-
+Our preferred method of patch submission is via a Git [pull request](https://help.github.com/articles/using-pull-requests).
-Our preferred method of patch submission is via a Git [pull request](https://help.github.com/articles/using-pull-requests).
+Many [people](https://raw.github.com/sqlmapproject/sqlmap/master/doc/THANKS.md) have contributed in different ways to the sqlmap development. **You** can be the next!
-Many [people](https://raw.github.com/sqlmapproject/sqlmap/master/doc/THANKS.md) have contributed in different ways to the sqlmap development. **You** can be the next!
+
-
+### Guidelines
-### Guidelines
+
-
+In order to maintain consistency and readability throughout the code, we ask that you adhere to the following instructions:
-In order to maintain consistency and readability throughout the code, we ask that you adhere to the following instructions:
+
-
+* Each patch should make one logical change.
-* Each patch should make one logical change.
+* Wrap code to 76 columns when possible.
-* Wrap code to 76 columns when possible.
+* Avoid tabbing, use four blank spaces instead.
-* Avoid tabbing, use four blank spaces instead.
+* Before you put time into a non-trivial patch, it is worth discussing it privately by [email](mailto:dev@sqlmap.org).
-* Before you put time into a non-trivial patch, it is worth discussing it on the [mailing list](https://lists.sourceforge.net/lists/listinfo/sqlmap-users) or privately by [email](mailto:dev@sqlmap.org).
+* Do not change style on numerous files in one single pull request, we can [discuss](mailto:dev@sqlmap.org) about those before doing any major restyling, but be sure that personal preferences not having a strong support in [PEP 8](http://www.python.org/dev/peps/pep-0008/) will likely to be rejected.
-* Do not change style on numerous files in one single pull request, we can [discuss](mailto:dev@sqlmap.org) about those before doing any major restyling, but be sure that personal preferences not having a strong support in [PEP 8](http://www.python.org/dev/peps/pep-0008/) will likely to be rejected.
+* Make changes on less than five files per single pull request - there is rarely a good reason to have more than five files changed on one pull request, as this dramatically increases the review time required to land (commit) any of those pull requests.
-* Make changes on less than five files per single pull request - there is rarely a good reason to have more than five files changed on one pull request, as this dramatically increases the review time required to land (commit) any of those pull requests.
+* Style that is too different from main branch will be ''adapted'' by the developers side.
-* Style that is too different from main branch will be ''adapted'' by the developers side.
+* Do not touch anything inside `thirdparty/` and `extra/` folders.
-* Do not touch anything inside `thirdparty/` and `extra/` folders.
+
-
+### Licensing
-### Licensing
+
-
+By submitting code contributions to the sqlmap developers or via Git pull request, checking them into the sqlmap source code repository, it is understood (unless you specify otherwise) that you are offering the sqlmap copyright holders the unlimited, non-exclusive right to reuse, modify, and relicense the code. This is important because the inability to relicense code has caused devastating problems for other software projects (such as KDE and NASM). If you wish to specify special license conditions of your contributions, just say so when you send them.
-By submitting code contributions to the sqlmap developers, to the mailing list, or via Git pull request, checking them into the sqlmap source code repository, it is understood (unless you specify otherwise) that you are offering the sqlmap copyright holders the unlimited, non-exclusive right to reuse, modify, and relicense the code. This is important because the inability to relicense code has caused devastating problems for other software projects (such as KDE and NASM). If you wish to specify special license conditions of your contributions, just say so when you send them.

doc/COPYING

@@ -1,7 +1,7 @@
 COPYING -- Describes the terms under which sqlmap is distributed. A copy
 of the GNU General Public License (GPL) is appended to this file.
 
-sqlmap is (C) 2006-2015 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar.
+sqlmap is (C) 2006-2017 Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar.
 
 This program is free software; you may redistribute and/or modify it under
 the terms of the GNU General Public License as published by the Free
@@ -46,14 +46,14 @@ to know exactly what a program is going to do before they run it.
 Source code also allows you to fix bugs and add new features. You are
 highly encouraged to send your changes to dev@sqlmap.org for possible
 incorporation into the main distribution. By sending these changes to the
-sqlmap developers, to the mailing lists, or via Git pull request, checking
+sqlmap developers or via Git pull request, checking them into the sqlmap
-them into the sqlmap source code repository, it is understood (unless you
+source code repository, it is understood (unless you specify otherwise)
-specify otherwise) that you are offering the sqlmap project the unlimited,
+that you are offering the sqlmap project the unlimited, non-exclusive
-non-exclusive right to reuse, modify, and relicense the code. sqlmap will
+right to reuse, modify, and relicense the code. sqlmap will always be
-always be available Open Source, but this is important because the
+available Open Source, but this is important because the inability to
-inability to relicense code has caused devastating problems for other Free
+relicense code has caused devastating problems for other Free Software
-Software projects (such as KDE and NASM). If you wish to specify special
+projects (such as KDE and NASM). If you wish to specify special license
-license conditions of your contributions, just say so when you send them.
+conditions of your contributions, just say so when you send them.
 
 This program is distributed in the hope that it will be useful, but
 WITHOUT ANY WARRANTY; without even the implied warranty of
@@ -361,7 +361,6 @@ This license does not apply to the following components:
 * The MultipartPost library located under thirdparty/multipartpost/.
 * The Odict library located under thirdparty/odict/.
 * The Oset library located under thirdparty/oset/.
-* The PageRank library located under thirdparty/pagerank/.
 * The PrettyPrint library located under thirdparty/prettyprint/.
 * The PyDes library located under thirdparty/pydes/.
 * The SocksiPy library located under thirdparty/socks/.

doc/THANKS.md

@@ -139,7 +139,7 @@ Jim Forster, <jimforster(at)goldenwest.com>
 * for reporting a bug
 
 Rong-En Fan, <rafan(at)freebsd.org>
-* for commiting the sqlmap 0.5 port to the official FreeBSD project repository
+* for committing the sqlmap 0.5 port to the official FreeBSD project repository
 
 Giorgio Fedon, <giorgio.fedon(at)gmail.com>
 * for suggesting a speed improvement for bisection algorithm
@@ -173,6 +173,9 @@ Ivan Giacomelli, <truemilk(at)insiberia.net>
 * for suggesting a minor enhancement
 * for reviewing the documentation
 
+Dimitris Giannitsaros, <daremon(at)gmail.com>
+* for contributing a REST-JSON API client
+
 Nico Golde, <nico(at)ngolde.de>
 * for reporting a couple of bugs
 
@@ -559,7 +562,7 @@ Kazim Bugra Tombul, <mhackmail(at)gmail.com>
 * for reporting a minor bug
 
 Efrain Torres, <et(at)metasploit.com>
-* for helping out to improve the Metasploit Framework sqlmap auxiliary module and for commiting it on the Metasploit official subversion repository
+* for helping out to improve the Metasploit Framework sqlmap auxiliary module and for committing it on the Metasploit official subversion repository
 * for his great Metasploit WMAP Framework
 
 Sandro Tosi, <matrixhasu(at)gmail.com>

doc/THIRD-PARTY.md

@@ -12,7 +12,7 @@ This file lists bundled packages and their associated licensing terms.
   Copyright (C) 2005, Zope Corporation.
   Copyright (C) 1998-2000, Gisle Aas.
 * The Colorama library located under thirdparty/colorama/.
-  Copyright (C) 2010, Jonathan Hartley.
+  Copyright (C) 2013, Jonathan Hartley.
 * The Fcrypt library located under thirdparty/fcrypt/.
   Copyright (C) 2000, 2001, 2004 Carey Evans.
 * The Odict library located under thirdparty/odict/.
@@ -281,8 +281,6 @@ be bound by the terms and conditions of this License Agreement.
 
 * The bottle web framework library located under thirdparty/bottle/.
   Copyright (C) 2012, Marcel Hellkamp.
-* The PageRank library located under thirdparty/pagerank/.
-  Copyright (C) 2010, Corey Goldberg.
 * The Termcolor library located under thirdparty/termcolor/.
   Copyright (C) 2008-2011, Volvox Development Team.
 
@@ -312,3 +310,5 @@ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
 
 * The PyDes library located under thirdparty/pydes/.
    Copyleft 2009, Todd Whiteman.
+* The win_inet_pton library located under thirdparty/wininetpton/.
+   Copyleft 2014, Ryan Vennell.

doc/translations/README-bg-BG.md

@@ -0,0 +1,50 @@
+# sqlmap
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![Лиценз](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+
+sqlmap e инструмент за тестване и проникване, с отворен код, който автоматизира процеса на откриване и използване на недостатъците на SQL база данните чрез SQL инжекция, която ги взима от сървъра. Снабден е с мощен детектор, множество специални функции за най-добрия тестер и широк спектър от функции, които могат да се използват за множество цели - извличане на данни от базата данни, достъп до основната файлова система и изпълняване на команди на операционната система.
+
+Демо снимки
+----
+
+![Снимка на екрана](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png)
+
+Можете да посетите [колекцията от снимки на екрана](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots), показващи някои  функции, качени на wiki.
+
+Инсталиране
+----
+
+Може да изтеглине най-новите tar архиви като кликнете [тук](https://github.com/sqlmapproject/sqlmap/tarball/master) или най-новите zip архиви като кликнете [тук](https://github.com/sqlmapproject/sqlmap/zipball/master).
+
+За предпочитане е да изтеглите sqlmap като клонирате [Git](https://github.com/sqlmapproject/sqlmap) хранилището:
+
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+
+sqlmap работи самостоятелно с [Python](http://www.python.org/download/) версия **2.6.x** и **2.7.x** на всички платформи.
+
+Използване
+----
+
+За да получите списък с основните опции използвайте:
+
+    python sqlmap.py -h
+
+За да получите списък с всички опции използвайте:
+
+    python sqlmap.py -hh
+
+Може да намерите пример за използване на sqlmap [тук](https://asciinema.org/a/46601).
+За да разберете възможностите на sqlmap, списък на поддържаните функции и описание на всички опции, заедно с примери, се препоръчва да се разгледа [упътването](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
+
+Връзки
+----
+
+* Начална страница: http://sqlmap.org
+* Изтегляне: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* RSS емисия: https://github.com/sqlmapproject/sqlmap/commits/master.atom
+* Проследяване на проблеми и въпроси: https://github.com/sqlmapproject/sqlmap/issues
+* Упътване: https://github.com/sqlmapproject/sqlmap/wiki
+* Често задавани въпроси (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+* Twitter: [@sqlmap](https://twitter.com/sqlmap)
+* Демо: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
+* Снимки на екрана: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-es-MX.md

@@ -0,0 +1,49 @@
+# sqlmap
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+
+sqlmap es una herramienta para pruebas de penetración "penetration testing" de software libre que automatiza el proceso de detección y explotación de fallos mediante inyección de SQL además de tomar el control de servidores de bases de datos. Contiene un poderoso motor de detección, así como muchas de las funcionalidades escenciales para el "pentester" y una amplia gama de opciones desde la recopilación de información para identificar el objetivo conocido como "fingerprinting" mediante la extracción de información de la base de datos, hasta el acceso al sistema de archivos subyacente para ejecutar comandos en el sistema operativo a través de conexiones alternativas conocidas como "Out-of-band".
+
+Capturas de Pantalla
+---
+![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png)
+
+Visita la [colección de capturas de pantalla](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) que demuestra algunas de las características en la documentación(wiki).
+
+Instalación
+---
+
+Se puede descargar el "tarball" más actual haciendo clic [aquí](https://github.com/sqlmapproject/sqlmap/tarball/master) o el "zipball" [aquí](https://github.com/sqlmapproject/sqlmap/zipball/master).
+
+Preferentemente, se puede descargar sqlmap clonando el repositorio [Git](https://github.com/sqlmapproject/sqlmap):
+
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+
+sqlmap funciona con las siguientes versiones de [Python](http://www.python.org/download/) ** 2.6.x** y ** 2.7.x** en cualquier plataforma.
+
+Uso
+---
+
+Para obtener una lista de opciones básicas: 
+
+    python sqlmap.py -h
+
+Para obtener una lista de todas las opciones:
+
+    python sqlmap.py -hh
+
+Se puede encontrar una muestra de su funcionamiento [aquí](https://asciinema.org/a/46601).
+Para obtener una visión general de las capacidades de sqlmap, así como un listado funciones soportadas y descripción de todas las opciones y modificadores, junto con ejemplos, se recomienda consultar el [manual de usuario](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
+
+Enlaces
+---
+
+* Página principal: http://sqlmap.org 
+* Descargar: [. tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) o [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* Fuente de Cambios "Commit RSS feed": https://github.com/sqlmapproject/sqlmap/commits/master.atom
+* Seguimiento de problemas "Issue tracker": https://github.com/sqlmapproject/sqlmap/issues
+* Manual de usuario: https://github.com/sqlmapproject/sqlmap/wiki
+* Preguntas frecuentes (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+* Twitter: [@sqlmap](https://twitter.com/sqlmap)
+* Demostraciones: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
+* Imágenes: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-fr-FR.md

@@ -0,0 +1,49 @@
+# sqlmap
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+
+**sqlmap** est un outil Open Source de test d'intrusion. Cet outil permet d'automatiser le processus de détection et d'exploitation des failles d'injection SQL afin de prendre le contrôle des serveurs de base de données. __sqlmap__ dispose d'un puissant moteur de détection utilisant les techniques les plus récentes et les plus dévastatrices de tests d'intrusion comme L'Injection SQL, qui permet d'accéder à la base de données, au système de fichiers sous-jacent et permet aussi l'exécution des commandes sur le système d'exploitation.
+
+----
+
+![Les Captures d'écran](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png)
+
+Les captures d'écran disponible [ici](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) démontrent des fonctionnalités de __sqlmap__.
+
+Installation
+----
+
+Vous pouvez télécharger le plus récent fichier tarball en cliquant [ici](https://github.com/sqlmapproject/sqlmap/tarball/master). Vous pouvez aussi télécharger le plus récent archive zip [ici](https://github.com/sqlmapproject/sqlmap/zipball/master).
+
+De préférence, télécharger __sqlmap__ en le [clonant](https://github.com/sqlmapproject/sqlmap):
+
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+
+sqlmap fonctionne sur n'importe quel système d'exploitation avec la version **2.6.x** et **2.7.x** de [Python](http://www.python.org/download/)  
+
+Usage
+----
+
+Pour afficher une liste des fonctions de bases et des commutateurs (switches), tapez:
+
+    python sqlmap.py -h
+
+Pour afficher une liste complète des options et des commutateurs (switches), tapez:
+
+    python sqlmap.py -hh
+
+Vous pouvez regarder un vidéo [ici](https://asciinema.org/a/46601) pour plus d'exemples.
+Pour obtenir un aperçu des ressources de __sqlmap__, une liste des fonctionnalités prises en charge et la description de toutes les options, ainsi que des exemples , nous vous recommandons de consulter [le wiki](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
+
+Liens
+----
+
+* Page d'acceuil: http://sqlmap.org
+* Téléchargement: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) ou [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* Commits RSS feed: https://github.com/sqlmapproject/sqlmap/commits/master.atom
+* Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
+* Manuel de l'utilisateur: https://github.com/sqlmapproject/sqlmap/wiki
+* Foire aux questions (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+* Twitter: [@sqlmap](https://twitter.com/sqlmap)
+* Démonstrations: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
+* Les captures d'écran: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-gr-GR.md

@@ -1,6 +1,6 @@
-sqlmap
+# sqlmap
-==
 
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 Το sqlmap είναι πρόγραμμα ανοιχτού κώδικα, που αυτοματοποιεί την εύρεση και εκμετάλλευση ευπαθειών τύπου SQL Injection σε βάσεις δεδομένων. Έρχεται με μια δυνατή μηχανή αναγνώρισης ευπαθειών, πολλά εξειδικευμένα χαρακτηριστικά για τον απόλυτο penetration tester όπως και με ένα μεγάλο εύρος επιλογών αρχίζοντας από την αναγνώριση της βάσης δεδομένων, κατέβασμα δεδομένων της βάσης, μέχρι και πρόσβαση στο βαθύτερο σύστημα αρχείων και εκτέλεση εντολών στο απευθείας στο λειτουργικό μέσω εκτός ζώνης συνδέσεων.
 
@@ -18,7 +18,7 @@ sqlmap
 
 Κατά προτίμηση, μπορείτε να κατεβάσετε το sqlmap κάνοντας κλώνο το [Git](https://github.com/sqlmapproject/sqlmap) αποθετήριο:
 
-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
 
 Το sqlmap λειτουργεί χωρίς περαιτέρω κόπο με την [Python](http://www.python.org/download/) έκδοσης **2.6.x** και **2.7.x** σε όποια πλατφόρμα.
 
@@ -33,8 +33,8 @@ sqlmap
 
     python sqlmap.py -hh
 
-Μπορείτε να δείτε ένα δείγμα λειτουργίας του προγράμματος [εδώ](https://gist.github.com/stamparm/5335217).
+Μπορείτε να δείτε ένα δείγμα λειτουργίας του προγράμματος [εδώ](https://asciinema.org/a/46601).
-Για μια γενικότερη άποψη των δυνατοτήτων του sqlmap, μια λίστα των υποστηριζόμενων χαρακτηριστικών και περιγραφή για όλες τις επιλογές, μαζί με παραδείγματα, καλείστε να συμβουλευτείτε το [εγχειρίδιο χρήστη](https://github.com/sqlmapproject/sqlmap/wiki).
+Για μια γενικότερη άποψη των δυνατοτήτων του sqlmap, μια λίστα των υποστηριζόμενων χαρακτηριστικών και περιγραφή για όλες τις επιλογές, μαζί με παραδείγματα, καλείστε να συμβουλευτείτε το [εγχειρίδιο χρήστη](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
 
 Σύνδεσμοι
 ----
@@ -45,9 +45,6 @@ sqlmap
 * Προβλήματα: https://github.com/sqlmapproject/sqlmap/issues
 * Εγχειρίδιο Χρήστη: https://github.com/sqlmapproject/sqlmap/wiki
 * Συχνές Ερωτήσεις (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
-* Εγγραφή σε Mailing list: https://lists.sourceforge.net/lists/listinfo/sqlmap-users
-* Mailing list RSS feed: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap
-* Mailing list αρχείο: http://news.gmane.org/gmane.comp.security.sqlmap
 * Twitter: [@sqlmap](https://twitter.com/sqlmap)
 * Demos: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
 * Εικόνες: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-hr-HR.md

@@ -1,6 +1,6 @@
-sqlmap
+# sqlmap
-==
 
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap je alat namijenjen za penetracijsko testiranje koji automatizira proces detekcije i eksploatacije sigurnosnih propusta SQL injekcije te preuzimanje poslužitelja baze podataka. Dolazi s moćnim mehanizmom za detekciju, mnoštvom korisnih opcija za napredno penetracijsko testiranje te široki spektar opcija od onih za prepoznavanja baze podataka, preko dohvaćanja podataka iz baze, do pristupa zahvaćenom datotečnom sustavu i izvršavanja komandi na operacijskom sustavu korištenjem tzv. "out-of-band" veza.
 
@@ -18,7 +18,7 @@ Možete preuzeti zadnji tarball klikom [ovdje](https://github.com/sqlmapproject/
 
 Po mogućnosti, možete preuzeti sqlmap kloniranjem [Git](https://github.com/sqlmapproject/sqlmap) repozitorija:
 
-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
 
 sqlmap radi bez posebnih zahtjeva korištenjem [Python](http://www.python.org/download/) verzije **2.6.x** i/ili **2.7.x** na bilo kojoj platformi.
 
@@ -33,8 +33,8 @@ Kako biste dobili listu svih opcija i prekidača koristite:
 
     python sqlmap.py -hh
 
-Možete pronaći primjer izvršavanja [ovdje](https://gist.github.com/stamparm/5335217).
+Možete pronaći primjer izvršavanja [ovdje](https://asciinema.org/a/46601).
-Kako biste dobili pregled mogućnosti sqlmap-a, liste podržanih značajki te opis svih opcija i prekidača, zajedno s primjerima, preporučen je uvid u [korisnički priručnik](https://github.com/sqlmapproject/sqlmap/wiki).
+Kako biste dobili pregled mogućnosti sqlmap-a, liste podržanih značajki te opis svih opcija i prekidača, zajedno s primjerima, preporučen je uvid u [korisnički priručnik](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
 
 Poveznice
 ----
@@ -45,9 +45,6 @@ Poveznice
 * Prijava problema: https://github.com/sqlmapproject/sqlmap/issues
 * Korisnički priručnik: https://github.com/sqlmapproject/sqlmap/wiki
 * Najčešće postavljena pitanja (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
-* Pretplata na mailing listu: https://lists.sourceforge.net/lists/listinfo/sqlmap-users
-* RSS feed mailing liste: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap
-* Arhiva mailing liste: http://news.gmane.org/gmane.comp.security.sqlmap
 * Twitter: [@sqlmap](https://twitter.com/sqlmap)
 * Demo: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
 * Slike zaslona: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-id-ID.md

@@ -1,5 +1,6 @@
-sqlmap
+# sqlmap
-==
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap merupakan alat _(tool)_ bantu _open source_ dalam melakukan tes penetrasi yang mengotomasi proses deteksi dan eksploitasi kelemahan _SQL injection_ dan pengambil-alihan server basisdata. sqlmap dilengkapi dengan pendeteksi canggih, fitur-fitur hanal bagi _penetration tester_, beragam cara untuk mendeteksi basisdata, hingga mengakses _file system_ dan mengeksekusi perintah dalam sistem operasi melalui koneksi _out-of-band_. 
 
@@ -18,7 +19,7 @@ Anda dapat mengunduh tarball versi terbaru [di sini]
 
 Sebagai alternatif, Anda dapat mengunduh sqlmap dengan men-_clone_ repositori [Git](https://github.com/sqlmapproject/sqlmap):
 
-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
 
 sqlmap berfungsi langsung pada [Python](http://www.python.org/download/) versi **2.6.x** dan **2.7.x** pada platform apapun.
 
@@ -33,8 +34,8 @@ Untuk mendapatkan daftar opsi lanjut gunakan:
 
     python sqlmap.py -hh
 
-Anda dapat mendapatkan contoh penggunaan [di sini](https://gist.github.com/stamparm/5335217).
+Anda dapat mendapatkan contoh penggunaan [di sini](https://asciinema.org/a/46601).
-Untuk mendapatkan gambaran singkat kemampuan sqlmap, daftar fitur yang didukung, deskripsi dari semua opsi, berikut dengan contohnya, Anda disarankan untuk membaca [manual pengguna](https://github.com/sqlmapproject/sqlmap/wiki).
+Untuk mendapatkan gambaran singkat kemampuan sqlmap, daftar fitur yang didukung, deskripsi dari semua opsi, berikut dengan contohnya, Anda disarankan untuk membaca [Panduan Pengguna](https://github.com/sqlmapproject/sqlmap/wiki/Usage).
 
 Tautan
 ----
@@ -45,9 +46,6 @@ Tautan
 * Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
 * Wiki Manual Penggunaan: https://github.com/sqlmapproject/sqlmap/wiki
 * Pertanyaan yang Sering Ditanyakan (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
-* Berlangganan milis: https://lists.sourceforge.net/lists/listinfo/sqlmap-users
-* RSS feed dari milis: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap
-* Arsip milis: http://news.gmane.org/gmane.comp.security.sqlmap
 * Twitter: [@sqlmap](https://twitter.com/sqlmap)
 * Video Demo [#1](http://www.youtube.com/user/inquisb/videos) dan [#2](http://www.youtube.com/user/stamparm/videos)
 * Tangkapan Layar: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-it-IT.md

@@ -0,0 +1,50 @@
+# sqlmap
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+
+sqlmap è uno strumento open source per il penetration testing. Il suo scopo è quello di rendere automatico il processo di scoperta ed exploit di vulnerabilità di tipo SQL injection al fine di compromettere database online. Dispone di un potente motore per la ricerca di vulnerabilità, molti strumenti di nicchia anche per il più esperto penetration tester ed un'ampia gamma di controlli che vanno dal fingerprinting di database allo scaricamento di dati, fino all'accesso al file system sottostante e l'esecuzione di comandi nel sistema operativo attraverso connessioni out-of-band.
+
+Screenshot
+----
+
+![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png)
+
+Nella wiki puoi visitare [l'elenco di screenshot](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) che mostrano il funzionamento di alcune delle funzionalità del programma.
+
+Installazione
+----
+
+Puoi scaricare l'ultima tarball cliccando [qui](https://github.com/sqlmapproject/sqlmap/tarball/master) oppure l'ultima zipball cliccando [qui](https://github.com/sqlmapproject/sqlmap/zipball/master).
+
+La cosa migliore sarebbe però scaricare sqlmap clonando la repository [Git](https://github.com/sqlmapproject/sqlmap):
+
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+
+sqlmap è in grado di funzionare con le versioni **2.6.x** e **2.7.x** di [Python](http://www.python.org/download/) su ogni piattaforma.
+
+Utilizzo
+----
+
+Per una lista delle opzioni e dei controlli di base:
+
+    python sqlmap.py -h
+
+Per una lista di tutte le opzioni e di tutti i controlli:
+
+    python sqlmap.py -hh
+
+Puoi trovare un esempio di esecuzione [qui](https://asciinema.org/a/46601).
+Per una panoramica delle capacità di sqlmap, una lista delle sue funzionalità e la descrizione di tutte le sue opzioni e controlli, insieme ad un gran numero di esempi, siete pregati di visitare lo [user's manual](https://github.com/sqlmapproject/sqlmap/wiki/Usage) (disponibile solo in inglese).
+
+Link
+----
+
+* Sito: http://sqlmap.org
+* Download: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* RSS feed dei commit: https://github.com/sqlmapproject/sqlmap/commits/master.atom
+* Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
+* Manuale dell'utente: https://github.com/sqlmapproject/sqlmap/wiki
+* Domande più frequenti (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+* Twitter: [@sqlmap](https://twitter.com/sqlmap)
+* Dimostrazioni: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
+* Screenshot: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-ja-JP.md

@@ -0,0 +1,51 @@
+# sqlmap
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+
+sqlmapはオープンソースのペネトレーションテスティングツールです。SQLインジェクションの脆弱性の検出、活用、そしてデータベースサーバ奪取のプロセスを自動化します。
+強力な検出エンジン、ペネトレーションテスターのための多くのニッチ機能、持続的なデータベースのフィンガープリンティングから、データベースのデータ取得やアウトオブバンド接続を介したオペレーティング・システム上でのコマンド実行、ファイルシステムへのアクセスなどの広範囲に及ぶスイッチを提供します。
+
+スクリーンショット
+----
+
+![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png)
+
+wikiに載っているいくつかの機能のデモをスクリーンショットで見ることができます。 [スクリーンショット集](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots)
+
+インストール
+----
+
+最新のtarballを [こちら](https://github.com/sqlmapproject/sqlmap/tarball/master) から、最新のzipballを [こちら](https://github.com/sqlmapproject/sqlmap/zipball/master) からダウンロードできます。
+
+[Git](https://github.com/sqlmapproject/sqlmap) レポジトリをクローンして、sqlmapをダウンロードすることも可能です。:
+
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+
+sqlmapは、 [Python](http://www.python.org/download/) バージョン **2.6.x** または **2.7.x** がインストールされていれば、全てのプラットフォームですぐに使用できます。
+
+使用法
+----
+
+基本的なオプションとスイッチの使用法をリストするには:
+
+    python sqlmap.py -h
+
+全てのオプションとスイッチの使用法をリストするには:
+
+    python sqlmap.py -hh
+
+実行例を [こちら](https://asciinema.org/a/46601) で見ることができます。
+sqlmapの概要、機能の一覧、全てのオプションやスイッチの使用法を例とともに、 [ユーザーマニュアル](https://github.com/sqlmapproject/sqlmap/wiki/Usage) で確認することができます。
+
+リンク
+----
+
+* ホームページ: http://sqlmap.org
+* ダウンロード: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* コミットのRSSフィード: https://github.com/sqlmapproject/sqlmap/commits/master.atom
+* 課題管理: https://github.com/sqlmapproject/sqlmap/issues
+* ユーザーマニュアル: https://github.com/sqlmapproject/sqlmap/wiki
+* よくある質問 (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+* Twitter: [@sqlmap](https://twitter.com/sqlmap)
+* デモ: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
+* スクリーンショット: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-pt-BR.md

@@ -1,5 +1,6 @@
-sqlmap
+# sqlmap
-==
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap é uma ferramenta de teste de penetração de código aberto que automatiza o processo de detecção e exploração de falhas de injeção SQL. Com essa ferramenta é possível assumir total controle de servidores de banco de dados em páginas web vulneráveis, inclusive de base de dados fora do sistema invadido. Ele possui um motor de detecção poderoso, empregando as últimas e mais devastadoras técnicas de teste de penetração por SQL Injection, que permite acessar a base de dados, o sistema de arquivos subjacente e executar comandos no sistema operacional.
 
@@ -18,7 +19,7 @@ Você pode baixar o arquivo tar mais recente clicando [aqui]
 
 De preferência, você pode baixar o sqlmap clonando o repositório [Git](https://github.com/sqlmapproject/sqlmap):
 
-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
 
 sqlmap funciona em [Python](http://www.python.org/download/) nas versões **2.6.x** e **2.7.x** em todas as plataformas.
 
@@ -33,7 +34,7 @@ Para obter a lista completa de opções faça:
 
     python sqlmap.py -hh
 
-Você pode encontrar alguns exemplos [aqui](https://gist.github.com/stamparm/5335217).
+Você pode encontrar alguns exemplos [aqui](https://asciinema.org/a/46601).
 Para ter uma visão geral dos recursos do sqlmap, lista de recursos suportados e a descrição de todas as opções, juntamente com exemplos, aconselhamos que você consulte o [manual do usuário](https://github.com/sqlmapproject/sqlmap/wiki).
 
 Links
@@ -45,9 +46,6 @@ Links
 * Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
 * Manual do Usuário: https://github.com/sqlmapproject/sqlmap/wiki
 * Perguntas frequentes (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
-* Mailing list subscription: https://lists.sourceforge.net/lists/listinfo/sqlmap-users
-* Mailing list RSS feed: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap
-* Mailing list archive: http://news.gmane.org/gmane.comp.security.sqlmap
 * Twitter: [@sqlmap](https://twitter.com/sqlmap)
 * Demonstrações: [#1](http://www.youtube.com/user/inquisb/videos) e [#2](http://www.youtube.com/user/stamparm/videos)
 * Imagens: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-tr-TR.md

@@ -0,0 +1,53 @@
+# sqlmap
+
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
+
+sqlmap sql injection açıklarını otomatik olarak tespit ve istismar etmeye yarayan açık kaynak bir penetrasyon aracıdır. sqlmap gelişmiş tespit özelliğinin yanı sıra penetrasyon testleri sırasında gerekli olabilecek bir çok aracı, -uzak veritabınınından, veri indirmek, dosya sistemine erişmek, dosya çalıştırmak gibi - işlevleri de barındırmaktadır.
+
+
+Ekran görüntüleri
+----
+
+![Screenshot](https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png)
+
+
+İsterseniz özelliklerin tanıtımının yapıldığı [collection of screenshots](https://github.com/sqlmapproject/sqlmap/wiki/Screenshots) sayfasını ziyaret edebilirsiniz.
+
+
+Kurulum
+----
+
+[Buraya](https://github.com/sqlmapproject/sqlmap/tarball/master) tıklayarak en son sürüm tarball'ı veya [buraya](https://github.com/sqlmapproject/sqlmap/zipball/master) tıklayarak zipbal'ı indirebilirsiniz.
+
+Veya tercihen, [Git](https://github.com/sqlmapproject/sqlmap) reposunu klonlayarak indirebilirsiniz
+
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+
+sqlmap [Python](http://www.python.org/download/) sitesinde bulunan **2.6.x** and **2.7.x** versiyonları ile bütün platformlarda çalışabilmektedir.
+
+Kullanım
+----
+
+
+Bütün basit seçeneklerin listesini gösterir
+
+    python sqlmap.py -h
+
+Bütün seçenekleri gösterir
+
+    python sqlmap.py -hh
+
+Program ile ilgili örnekleri [burada](https://asciinema.org/a/46601) bulabilirsiniz. Daha fazlası içinsqlmap'in bütün açıklamaları ile birlikte bütün özelliklerinin, örnekleri ile bulunduğu  [manuel sayfamıza](https://github.com/sqlmapproject/sqlmap/wiki/Usage) bakmanızı tavsiye ediyoruz
+
+Links
+----
+
+* Anasayfa: http://sqlmap.org
+* İndirme bağlantıları: [.tar.gz](https://github.com/sqlmapproject/sqlmap/tarball/master) or [.zip](https://github.com/sqlmapproject/sqlmap/zipball/master)
+* Commitlerin RSS beslemeleri: https://github.com/sqlmapproject/sqlmap/commits/master.atom
+* Hata takip etme sistemi: https://github.com/sqlmapproject/sqlmap/issues
+* Kullanıcı Manueli: https://github.com/sqlmapproject/sqlmap/wiki
+* Sıkça Sorulan Sorular(SSS): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+* Twitter: [@sqlmap](https://twitter.com/sqlmap)
+* Demolar: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
+* Ekran görüntüleri: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

doc/translations/README-zh-CN.md

@@ -1,6 +1,6 @@
-sqlmap
+# sqlmap
-==
 
+[![Build Status](https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master)](https://api.travis-ci.org/sqlmapproject/sqlmap) [![Python 2.6|2.7](https://img.shields.io/badge/python-2.6|2.7-yellow.svg)](https://www.python.org/) [![License](https://img.shields.io/badge/license-GPLv2-red.svg)](https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING) [![Twitter](https://img.shields.io/badge/twitter-@sqlmap-blue.svg)](https://twitter.com/sqlmap)
 
 sqlmap 是一个开源的渗透测试工具，可以用来自动化的检测，利用SQL注入漏洞，获取数据库服务器的权限。它具有功能强大的检测引擎,针对各种不同类型数据库的渗透测试的功能选项，包括获取数据库中存储的数据，访问操作系统文件甚至可以通过外带数据连接的方式执行操作系统命令。
 
@@ -18,7 +18,7 @@ sqlmap 是一个开源的渗透测试工具，可以用来自动化的检测，
 
 推荐你从 [Git](https://github.com/sqlmapproject/sqlmap) 仓库获取最新的源代码:
 
-    git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
 
 sqlmap 可以运行在 [Python](http://www.python.org/download/)  **2.6.x**  和  **2.7.x** 版本的任何平台上
 
@@ -33,7 +33,7 @@ sqlmap 可以运行在 [Python](http://www.python.org/download/)  **2.6.x**  和
 
     python sqlmap.py -hh
 
-你可以从 [这里](https://gist.github.com/stamparm/5335217) 看到一个sqlmap 的使用样例。除此以外，你还可以查看 [使用手册](https://github.com/sqlmapproject/sqlmap/wiki)。获取sqlmap所有支持的特性、参数、命令行选项开关及说明的使用帮助。
+你可以从 [这里](https://asciinema.org/a/46601) 看到一个sqlmap 的使用样例。除此以外，你还可以查看 [使用手册](https://github.com/sqlmapproject/sqlmap/wiki/Usage)。获取sqlmap所有支持的特性、参数、命令行选项开关及说明的使用帮助。
 
 链接
 ----
@@ -44,9 +44,6 @@ sqlmap 可以运行在 [Python](http://www.python.org/download/)  **2.6.x**  和
 * Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
 * 使用手册: https://github.com/sqlmapproject/sqlmap/wiki
 * 常见问题 (FAQ): https://github.com/sqlmapproject/sqlmap/wiki/FAQ
-* 邮件讨论列表: https://lists.sourceforge.net/lists/listinfo/sqlmap-users
-* 邮件列表 RSS 订阅: http://rss.gmane.org/messages/complete/gmane.comp.security.sqlmap
-* 邮件列表归档: http://news.gmane.org/gmane.comp.security.sqlmap
 * Twitter: [@sqlmap](https://twitter.com/sqlmap)
 * 教程: [http://www.youtube.com/user/inquisb/videos](http://www.youtube.com/user/inquisb/videos)
 * 截图: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots

extra/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

extra/beep/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

extra/beep/beep.py

@@ -3,7 +3,7 @@
 """
 beep.py - Make a beep sound
 
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

extra/cloak/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

extra/cloak/cloak.py

@@ -3,7 +3,7 @@
 """
 cloak.py - Simple file encryption/compression utility
 
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -24,17 +24,19 @@ def hideAscii(data):
 
     return retVal
 
-def cloak(inputFile):
+def cloak(inputFile=None, data=None):
-    f = open(inputFile, 'rb')
+    if data is None:
-    data = zlib.compress(f.read())
+        with open(inputFile, "rb") as f:
-    f.close()
+            data = f.read()
 
-    return hideAscii(data)
+    return hideAscii(zlib.compress(data))
 
-def decloak(inputFile):
+def decloak(inputFile=None, data=None):
-    f = open(inputFile, 'rb')
+    if data is None:
+        with open(inputFile, "rb") as f:
+            data = f.read()
     try:
-        data = zlib.decompress(hideAscii(f.read()))
+        data = zlib.decompress(hideAscii(data))
     except:
         print 'ERROR: the provided input file \'%s\' does not contain valid cloaked content' % inputFile
         sys.exit(1)

extra/dbgtool/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

extra/dbgtool/dbgtool.py

@@ -3,7 +3,7 @@
 """
 dbgtool.py - Portable executable to ASCII debug script converter
 
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

extra/icmpsh/icmpsh-s.c

@@ -99,7 +99,7 @@ void usage(char *path)
 	printf("  -h                 this screen\n");
 	printf("  -b num             maximal number of blanks (unanswered icmp requests)\n");
     printf("                     before quitting\n");
-	printf("  -s bytes           maximal data buffer size in bytes (default is 64 bytes)\n\n", DEFAULT_MAX_DATA_SIZE);
+	printf("  -s bytes           maximal data buffer size in bytes (default is %u bytes)\n\n", DEFAULT_MAX_DATA_SIZE);
 	printf("In order to improve the speed, lower the delay (-d) between requests or\n");
     printf("increase the size (-s) of the data buffer\n");
 }
@@ -203,8 +203,6 @@ int main(int argc, char **argv)
 	PROCESS_INFORMATION pi;
 	int status;
 	unsigned int max_data_size;
-	struct hostent *he;
-
 
 	// set defaults
 	target = 0;

extra/icmpsh/icmpsh_m.py

@@ -76,7 +76,7 @@ def main(src, dst):
     # Instantiate an IP packets decoder
     decoder = ImpactDecoder.IPDecoder()
 
-    while 1:
+    while True:
         cmd = ''
 
         # Wait for incoming replies

extra/mssqlsig/update.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -43,7 +43,7 @@ def updateMSSQLXML():
 
         return
 
-    releases = re.findall("class=\"BCC_DV_01DarkBlueTitle\">SQL Server\s(.+?)\sBuilds", mssqlVersionsHtmlString, re.I | re.M)
+    releases = re.findall("class=\"BCC_DV_01DarkBlueTitle\">SQL Server\s(.+?)\sBuilds", mssqlVersionsHtmlString, re.I)
     releasesCount = len(releases)
 
     # Create the minidom document
@@ -74,7 +74,7 @@ def updateMSSQLXML():
             stopIdx = mssqlVersionsHtmlString.index("SQL Server %s Builds" % releases[index + 1])
 
         mssqlVersionsReleaseString = mssqlVersionsHtmlString[startIdx:stopIdx]
-        servicepackVersion = re.findall("</td><td>[7\.0|2000|2005|2008|2008 R2]*(.*?)</td><td.*?([\d\.]+)</td>[\r]*\n", mssqlVersionsReleaseString, re.I | re.M)
+        servicepackVersion = re.findall("</td><td>(7\.0|2000|2005|2008|2008 R2)*(.*?)</td><td.*?([\d\.]+)</td>[\r]*\n", mssqlVersionsReleaseString, re.I)
 
         for servicePack, version in servicepackVersion:
             if servicePack.startswith(" "):

extra/runcmd/README.txt

@@ -1,3 +1,3 @@
-Files in this folder can be used to compile auxiliary program that can
+runcmd.exe is an auxiliary program that can be used for running command prompt
-be used for running command prompt commands skipping standard "cmd /c" way. 
+commands skipping standard "cmd /c" way. It is licensed under the terms of the
-They are licensed under the terms of the GNU Lesser General Public License.
+GNU Lesser General Public License.

extra/safe2bin/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

extra/safe2bin/safe2bin.py

@@ -3,7 +3,7 @@
 """
 safe2bin.py - Simple safe(hex) to binary format converter
 
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -19,14 +19,17 @@ from optparse import OptionParser
 # Regex used for recognition of hex encoded characters
 HEX_ENCODED_CHAR_REGEX = r"(?P<result>\\x[0-9A-Fa-f]{2})"
 
-# Regex used for recognition of representation for hex encoded invalid unicode characters
-INVALID_UNICODE_CHAR_REGEX = r"(?P<result>\\\?[0-9A-Fa-f]{2})"
-
 # Raw chars that will be safe encoded to their slash (\) representations (e.g. newline to \n)
 SAFE_ENCODE_SLASH_REPLACEMENTS = "\t\n\r\x0b\x0c"
 
 # Characters that don't need to be safe encoded
-SAFE_CHARS = "".join(filter(lambda x: x not in SAFE_ENCODE_SLASH_REPLACEMENTS, string.printable.replace('\\', '')))
+SAFE_CHARS = "".join(filter(lambda _: _ not in SAFE_ENCODE_SLASH_REPLACEMENTS, string.printable.replace('\\', '')))
+
+# Prefix used for hex encoded values
+HEX_ENCODED_PREFIX = r"\x"
+
+# Strings used for temporary marking of hex encoded prefixes (to prevent double encoding)
+HEX_ENCODED_PREFIX_MARKER = "__HEX_ENCODED_PREFIX__"
 
 # String used for temporary marking of slash characters
 SLASH_MARKER = "__SLASH__"
@@ -44,7 +47,8 @@ def safecharencode(value):
     retVal = value
 
     if isinstance(value, basestring):
-        if any(_ not in SAFE_CHARS for _ in value):
+        if any([_ not in SAFE_CHARS for _ in value]):
+            retVal = retVal.replace(HEX_ENCODED_PREFIX, HEX_ENCODED_PREFIX_MARKER)
             retVal = retVal.replace('\\', SLASH_MARKER)
 
             for char in SAFE_ENCODE_SLASH_REPLACEMENTS:
@@ -53,6 +57,7 @@ def safecharencode(value):
             retVal = reduce(lambda x, y: x + (y if (y in string.printable or isinstance(value, unicode) and ord(y) >= 160) else '\\x%02x' % ord(y)), retVal, (unicode if isinstance(value, unicode) else str)())
 
             retVal = retVal.replace(SLASH_MARKER, "\\\\")
+            retVal = retVal.replace(HEX_ENCODED_PREFIX_MARKER, HEX_ENCODED_PREFIX)
     elif isinstance(value, list):
         for i in xrange(len(value)):
             retVal[i] = safecharencode(value[i])
@@ -83,12 +88,6 @@ def safechardecode(value, binary=False):
         if binary:
             if isinstance(retVal, unicode):
                 retVal = retVal.encode("utf8")
-            while True:
-                match = re.search(INVALID_UNICODE_CHAR_REGEX, retVal)
-                if match:
-                    retVal = retVal.replace(match.group("result"), chr(ord(binascii.unhexlify(match.group("result").lstrip("\\?")))))
-                else:
-                    break
 
     elif isinstance(value, (list, tuple)):
         for i in xrange(len(value)):

extra/shutils/duplicates.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 
-# Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 # See the file 'doc/COPYING' for copying permission
 
 # Removes duplicate entries in wordlist like files

extra/shutils/postcommit-hook.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+
+SETTINGS="../../lib/core/settings.py"
+
+declare -x SCRIPTPATH="${0}"
+
+FULLPATH=${SCRIPTPATH%/*}/$SETTINGS
+
+if [ -f $FULLPATH ]
+then
+    LINE=$(grep -o ${FULLPATH} -e 'VERSION = "[0-9.]*"')
+    declare -a LINE
+    NEW_TAG=$(python -c "import re, sys, time; version = re.search('\"([0-9.]*)\"', sys.argv[1]).group(1); _ = version.split('.'); print '.'.join(_[:-1]) if len(_) == 4 and _[-1] == '0' else ''" "$LINE")
+    if [ -n "$NEW_TAG" ]
+    then
+        #git commit -am "Automatic monthly tagging"
+        echo "Creating new tag ${NEW_TAG}"
+        git tag $NEW_TAG
+        git push origin $NEW_TAG
+        echo "Going to push PyPI package"
+        /bin/bash ${SCRIPTPATH%/*}/pypi.sh
+    fi
+fi

extra/shutils/precommit-hook.sh

@@ -0,0 +1,32 @@
+#!/bin/bash
+
+PROJECT="../../"
+SETTINGS="../../lib/core/settings.py"
+CHECKSUM="../../txt/checksum.md5"
+
+declare -x SCRIPTPATH="${0}"
+
+PROJECT_FULLPATH=${SCRIPTPATH%/*}/$PROJECT
+SETTINGS_FULLPATH=${SCRIPTPATH%/*}/$SETTINGS
+CHECKSUM_FULLPATH=${SCRIPTPATH%/*}/$CHECKSUM
+
+git diff $SETTINGS_FULLPATH | grep "VERSION =" > /dev/null && exit 0
+
+if [ -f $SETTINGS_FULLPATH ]
+then
+    LINE=$(grep -o ${SETTINGS_FULLPATH} -e 'VERSION = "[0-9.]*"')
+    declare -a LINE
+    INCREMENTED=$(python -c "import re, sys, time; version = re.search('\"([0-9.]*)\"', sys.argv[1]).group(1); _ = version.split('.'); _.append(0) if len(_) < 3 else _; _[-1] = str(int(_[-1]) + 1); month = str(time.gmtime().tm_mon); _[-1] = '0' if _[-2] != month else _[-1]; _[-2] = month; print sys.argv[1].replace(version, '.'.join(_))" "$LINE")
+    if [ -n "$INCREMENTED" ]
+    then
+        sed -i "s/${LINE}/${INCREMENTED}/" $SETTINGS_FULLPATH
+        echo "Updated ${INCREMENTED} in ${SETTINGS_FULLPATH}"
+    else
+        echo "Something went wrong in VERSION increment"
+        exit 1
+    fi
+    git add "$SETTINGS_FULLPATH"
+fi
+
+truncate -s 0 "$CHECKSUM_FULLPATH"
+cd $PROJECT_FULLPATH && for i in $(find . -name "*.py" -o -name "*.xml" -o -iname "*_" | sort); do git ls-files $i --error-unmatch &>/dev/null && md5sum $i | stdbuf -i0 -o0 -e0 sed 's/\.\///' >> "$CHECKSUM_FULLPATH"; git add "$CHECKSUM_FULLPATH"; done

extra/shutils/pydiatra.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+# Copyright (c) 2006-2013 sqlmap developers (http://sqlmap.org/)
+# See the file 'doc/COPYING' for copying permission
+
+# Runs py2diatra on all python files (prerequisite: pip install pydiatra)
+find . -wholename "./thirdparty" -prune -o -type f -iname "*.py" -exec py2diatra '{}' \; | grep -v bare-except

extra/shutils/pylint.py

@@ -20,8 +20,8 @@ def check(module):
         print "CHECKING ", module
         pout = os.popen("pylint --rcfile=/dev/null %s" % module, 'r')
         for line in pout:
-            if  re.match("E....:.", line):
+            if  re.match("\AE:", line):
-                print line
+                print line.strip()
             if __RATING__ and "Your code has been rated at" in line:
                 print line
                 score = re.findall("\d.\d\d", line)[0]

extra/shutils/pypi.sh

@@ -0,0 +1,171 @@
+#!/bin/bash
+
+declare -x SCRIPTPATH="${0}"
+SETTINGS="${SCRIPTPATH%/*}/../../lib/core/settings.py"
+VERSION=$(cat $SETTINGS | grep -E "^VERSION =" | cut -d '"' -f 2 | cut -d '.' -f 1-3)
+TYPE=pip
+TMP_DIR=/tmp/pypi
+mkdir $TMP_DIR
+cd $TMP_DIR
+cat > $TMP_DIR/setup.py << EOF
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+from setuptools import setup, find_packages
+
+setup(
+    name='sqlmap',
+    version='$VERSION',
+    description="Automatic SQL injection and database takeover tool",
+    author='Bernardo Damele Assumpcao Guimaraes, Miroslav Stampar',
+    author_email='bernardo@sqlmap.org, miroslav@sqlmap.org',
+    url='https://sqlmap.org',
+    download_url='https://github.com/sqlmapproject/sqlmap/archive/$VERSION.zip',
+    license='GNU General Public License v2 (GPLv2)',
+    packages=find_packages(),
+    include_package_data=True,
+    zip_safe=False,
+    # https://pypi.python.org/pypi?%3Aaction=list_classifiers
+    classifiers=[
+        'Development Status :: 5 - Production/Stable',
+        'License :: OSI Approved :: GNU General Public License v2 (GPLv2)',
+        'Natural Language :: English',
+        'Operating System :: OS Independent',
+        'Programming Language :: Python',
+        'Environment :: Console',
+        'Topic :: Database',
+        'Topic :: Security',
+    ],
+    entry_points={
+        'console_scripts': [
+            'sqlmap = sqlmap.sqlmap:main',
+        ],
+    },
+)
+EOF
+wget "https://github.com/sqlmapproject/sqlmap/archive/$VERSION.zip" -O sqlmap.zip
+unzip sqlmap.zip
+rm sqlmap.zip
+mv "sqlmap-$VERSION" sqlmap
+cat > sqlmap/__init__.py << EOF
+#!/usr/bin/env python
+
+"""
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
+See the file 'doc/COPYING' for copying permission
+"""
+
+import os
+import sys
+
+sys.dont_write_bytecode = True
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+EOF
+cat > README.rst << "EOF"
+sqlmap
+======
+
+|Build Status| |Python 2.6|2.7| |License| |Twitter|
+
+sqlmap is an open source penetration testing tool that automates the
+process of detecting and exploiting SQL injection flaws and taking over
+of database servers. It comes with a powerful detection engine, many
+niche features for the ultimate penetration tester and a broad range of
+switches lasting from database fingerprinting, over data fetching from
+the database, to accessing the underlying file system and executing
+commands on the operating system via out-of-band connections.
+
+Screenshots
+-----------
+
+.. figure:: https://raw.github.com/wiki/sqlmapproject/sqlmap/images/sqlmap_screenshot.png
+   :alt: Screenshot
+
+
+You can visit the `collection of
+screenshots <https://github.com/sqlmapproject/sqlmap/wiki/Screenshots>`__
+demonstrating some of features on the wiki.
+
+Installation
+------------
+
+You can use pip to install and/or upgrade the sqlmap to latest (monthly) tagged version with: ::
+
+    pip install --upgrade sqlmap
+
+Alternatively, you can download the latest tarball by clicking
+`here <https://github.com/sqlmapproject/sqlmap/tarball/master>`__ or
+latest zipball by clicking
+`here <https://github.com/sqlmapproject/sqlmap/zipball/master>`__.
+
+If you prefer fetching daily updates, you can download sqlmap by cloning the
+`Git <https://github.com/sqlmapproject/sqlmap>`__ repository:
+
+::
+
+    git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap-dev
+
+sqlmap works out of the box with
+`Python <http://www.python.org/download/>`__ version **2.6.x** and
+**2.7.x** on any platform.
+
+Usage
+-----
+
+To get a list of basic options and switches use:
+
+::
+
+    python sqlmap.py -h
+
+To get a list of all options and switches use:
+
+::
+
+    python sqlmap.py -hh
+
+You can find a sample run `here <https://asciinema.org/a/46601>`__. To
+get an overview of sqlmap capabilities, list of supported features and
+description of all options and switches, along with examples, you are
+advised to consult the `user's
+manual <https://github.com/sqlmapproject/sqlmap/wiki/Usage>`__.
+
+Links
+-----
+
+-  Homepage: http://sqlmap.org
+-  Download:
+   `.tar.gz <https://github.com/sqlmapproject/sqlmap/tarball/master>`__
+   or `.zip <https://github.com/sqlmapproject/sqlmap/zipball/master>`__
+-  Commits RSS feed:
+   https://github.com/sqlmapproject/sqlmap/commits/master.atom
+-  Issue tracker: https://github.com/sqlmapproject/sqlmap/issues
+-  User's manual: https://github.com/sqlmapproject/sqlmap/wiki
+-  Frequently Asked Questions (FAQ):
+   https://github.com/sqlmapproject/sqlmap/wiki/FAQ
+-  Twitter: [@sqlmap](https://twitter.com/sqlmap)
+-  Demos: http://www.youtube.com/user/inquisb/videos
+-  Screenshots: https://github.com/sqlmapproject/sqlmap/wiki/Screenshots
+
+.. |Build Status| image:: https://api.travis-ci.org/sqlmapproject/sqlmap.svg?branch=master
+   :target: https://api.travis-ci.org/sqlmapproject/sqlmap
+.. |Python 2.6|2.7| image:: https://img.shields.io/badge/python-2.6|2.7-yellow.svg
+   :target: https://www.python.org/
+.. |License| image:: https://img.shields.io/badge/license-GPLv2-red.svg
+   :target: https://raw.githubusercontent.com/sqlmapproject/sqlmap/master/doc/COPYING
+.. |Twitter| image:: https://img.shields.io/badge/twitter-@sqlmap-blue.svg
+   :target: https://twitter.com/sqlmap
+
+.. pandoc --from=markdown --to=rst --output=README.rst sqlmap/README.md
+.. http://rst.ninjs.org/
+EOF
+sed -i "s/^VERSION =.*/VERSION = \"$VERSION\"/g" sqlmap/lib/core/settings.py
+sed -i "s/^TYPE =.*/TYPE = \"$TYPE\"/g" sqlmap/lib/core/settings.py
+sed -i "s/.*lib\/core\/settings\.py/`md5sum sqlmap/lib/core/settings.py | cut -d ' ' -f 1`  lib\/core\/settings\.py/g" sqlmap/txt/checksum.md5
+for file in $(find sqlmap -type f | grep -v -E "\.(git|yml)"); do echo include $file >> MANIFEST.in; done
+python setup.py sdist upload
+rm -rf $TMP_DIR

extra/shutils/regressiontest.py

@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 
-# Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+# Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 # See the file 'doc/COPYING' for copying permission
 
 import codecs
@@ -22,7 +22,6 @@ from lib.core.revision import getRevisionNumber
 
 START_TIME = time.strftime("%H:%M:%S %d-%m-%Y", time.gmtime())
 SQLMAP_HOME = "/opt/sqlmap"
-REVISION = getRevisionNumber()
 
 SMTP_SERVER = "127.0.0.1"
 SMTP_PORT = 25
@@ -30,7 +29,7 @@ SMTP_TIMEOUT = 30
 FROM = "regressiontest@sqlmap.org"
 #TO = "dev@sqlmap.org"
 TO = ["bernardo.damele@gmail.com", "miroslav.stampar@gmail.com"]
-SUBJECT = "regression test started on %s using revision %s" % (START_TIME, REVISION)
+SUBJECT = "regression test started on %s using revision %s" % (START_TIME, getRevisionNumber())
 TARGET = "debian"
 
 def prepare_email(content):
@@ -41,7 +40,7 @@ def prepare_email(content):
     msg = MIMEMultipart()
     msg["Subject"] = SUBJECT
     msg["From"] = FROM
-    msg["To"] = TO if isinstance(TO, basestring) else ",".join(TO)
+    msg["To"] = TO if isinstance(TO, basestring) else ','.join(TO)
 
     msg.attach(MIMEText(content))
 
@@ -84,7 +83,7 @@ def main():
     if stderr:
         failure_email("Execution of regression test failed with error:\n\n%s" % stderr)
 
-    failed_tests = re.findall("running live test case: (.+?) \((\d+)\/\d+\)[\r]*\n.+test failed (at parsing items: (.+))?\s*\- scan folder: (\/.+) \- traceback: (.*?)( - SQL injection not detected)?[\r]*\n", stdout, re.M)
+    failed_tests = re.findall("running live test case: (.+?) \((\d+)\/\d+\)[\r]*\n.+test failed (at parsing items: (.+))?\s*\- scan folder: (\/.+) \- traceback: (.*?)( - SQL injection not detected)?[\r]*\n", stdout)
 
     for failed_test in failed_tests:
         title = failed_test[0]

extra/shutils/strip.sh

@@ -0,0 +1,15 @@
+#!/bin/bash
+
+# References:   http://www.thegeekstuff.com/2012/09/strip-command-examples/
+#               http://www.muppetlabs.com/~breadbox/software/elfkickers.html
+#               https://ptspts.blogspot.hr/2013/12/how-to-make-smaller-c-and-c-binaries.html
+
+# For example:
+# python ../../../../../extra/cloak/cloak.py -d -i lib_postgresqludf_sys.so_
+# ../../../../../extra/shutils/strip.sh lib_postgresqludf_sys.so
+# python ../../../../../extra/cloak/cloak.py -i lib_postgresqludf_sys.so
+# rm lib_postgresqludf_sys.so
+
+strip -S --strip-unneeded --remove-section=.note.gnu.gold-version --remove-section=.comment --remove-section=.note --remove-section=.note.gnu.build-id --remove-section=.note.ABI-tag $*
+sstrip $*
+

extra/sqlharvest/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

extra/sqlharvest/sqlharvest.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/controller/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/controller/action.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -16,8 +16,8 @@ from lib.core.enums import CONTENT_TYPE
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.exception import SqlmapUnsupportedDBMSException
 from lib.core.settings import SUPPORTED_DBMS
-from lib.techniques.brute.use import columnExists
+from lib.utils.brute import columnExists
-from lib.techniques.brute.use import tableExists
+from lib.utils.brute import tableExists
 
 def action():
     """
@@ -48,9 +48,6 @@ def action():
         elif kb.nullConnection:
             errMsg += ". You can try to rerun without using optimization "
             errMsg += "switch '%s'" % ("-o" if conf.optimize else "--null-connection")
-        else:
-            errMsg += ". Support for this DBMS will be implemented at "
-            errMsg += "some point"
 
         raise SqlmapUnsupportedDBMSException(errMsg)
 
@@ -77,8 +74,7 @@ def action():
 
     if conf.getPasswordHashes:
         try:
-            conf.dumper.userSettings("database management system users password hashes",
+            conf.dumper.userSettings("database management system users password hashes", conf.dbmsHandler.getPasswordHashes(), "password hash", CONTENT_TYPE.PASSWORDS)
-                                    conf.dbmsHandler.getPasswordHashes(), "password hash", CONTENT_TYPE.PASSWORDS)
         except SqlmapNoneDataException, ex:
             logger.critical(ex)
         except:
@@ -86,8 +82,7 @@ def action():
 
     if conf.getPrivileges:
         try:
-            conf.dumper.userSettings("database management system users privileges",
+            conf.dumper.userSettings("database management system users privileges", conf.dbmsHandler.getPrivileges(), "privilege", CONTENT_TYPE.PRIVILEGES)
-                                    conf.dbmsHandler.getPrivileges(), "privilege", CONTENT_TYPE.PRIVILEGES)
         except SqlmapNoneDataException, ex:
             logger.critical(ex)
         except:
@@ -95,8 +90,7 @@ def action():
 
     if conf.getRoles:
         try:
-            conf.dumper.userSettings("database management system users roles",
+            conf.dumper.userSettings("database management system users roles", conf.dbmsHandler.getRoles(), "role", CONTENT_TYPE.ROLES)
-                                    conf.dbmsHandler.getRoles(), "role", CONTENT_TYPE.ROLES)
         except SqlmapNoneDataException, ex:
             logger.critical(ex)
         except:

lib/controller/checks.py

@@ -1,18 +1,18 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
 import copy
 import httplib
+import random
 import re
 import socket
+import subprocess
 import time
 
-from subprocess import Popen as execute
-
 from extra.beep.beep import beep
 from lib.core.agent import agent
 from lib.core.common import Backend
@@ -20,10 +20,14 @@ from lib.core.common import extractRegexResult
 from lib.core.common import extractTextTagContent
 from lib.core.common import findDynamicContent
 from lib.core.common import Format
+from lib.core.common import getFilteredPageContent
 from lib.core.common import getLastRequestHTTPError
 from lib.core.common import getPublicTypeMembers
+from lib.core.common import getSafeExString
 from lib.core.common import getSortedInjectionTests
 from lib.core.common import getUnicode
+from lib.core.common import hashDBRetrieve
+from lib.core.common import hashDBWrite
 from lib.core.common import intersect
 from lib.core.common import listToStrValue
 from lib.core.common import parseFilePaths
@@ -38,6 +42,7 @@ from lib.core.common import singleTimeWarnMessage
 from lib.core.common import urlencode
 from lib.core.common import wasLastResponseDBMSError
 from lib.core.common import wasLastResponseHTTPError
+from lib.core.defaults import defaults
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
@@ -46,9 +51,11 @@ from lib.core.datatype import InjectionDict
 from lib.core.decorators import cachedmethod
 from lib.core.dicts import FROM_DUMMY_TABLE
 from lib.core.enums import DBMS
+from lib.core.enums import HASHDB_KEYS
 from lib.core.enums import HEURISTIC_TEST
 from lib.core.enums import HTTP_HEADER
 from lib.core.enums import HTTPMETHOD
+from lib.core.enums import NOTE
 from lib.core.enums import NULLCONNECTION
 from lib.core.enums import PAYLOAD
 from lib.core.enums import PLACE
@@ -57,18 +64,27 @@ from lib.core.exception import SqlmapConnectionException
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.exception import SqlmapSilentQuitException
 from lib.core.exception import SqlmapUserQuitException
+from lib.core.settings import CANDIDATE_SENTENCE_MIN_LENGTH
+from lib.core.settings import CHECK_INTERNET_ADDRESS
+from lib.core.settings import CHECK_INTERNET_VALUE
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
-from lib.core.settings import DUMMY_XSS_CHECK_APPENDIX
+from lib.core.settings import DUMMY_NON_SQLI_CHECK_APPENDIX
+from lib.core.settings import FI_ERROR_REGEX
 from lib.core.settings import FORMAT_EXCEPTION_STRINGS
 from lib.core.settings import HEURISTIC_CHECK_ALPHABET
+from lib.core.settings import IDS_WAF_CHECK_PAYLOAD
+from lib.core.settings import IDS_WAF_CHECK_RATIO
+from lib.core.settings import IDS_WAF_CHECK_TIMEOUT
+from lib.core.settings import MAX_DIFFLIB_SEQUENCE_LENGTH
+from lib.core.settings import NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH
+from lib.core.settings import SLEEP_TIME_MARKER
 from lib.core.settings import SUHOSIN_MAX_VALUE_LENGTH
 from lib.core.settings import SUPPORTED_DBMS
 from lib.core.settings import URI_HTTP_HEADER
 from lib.core.settings import UPPER_RATIO_BOUND
-from lib.core.settings import IDS_WAF_CHECK_PAYLOAD
-from lib.core.settings import IDS_WAF_CHECK_RATIO
 from lib.core.threads import getCurrentThreadData
 from lib.request.connect import Connect as Request
+from lib.request.comparison import comparison
 from lib.request.inject import checkBooleanExpression
 from lib.request.templates import getPageTemplate
 from lib.techniques.union.test import unionTest
@@ -82,6 +98,13 @@ def checkSqlInjection(place, parameter, value):
     # Localized thread data needed for some methods
     threadData = getCurrentThreadData()
 
+    # Favoring non-string specific boundaries in case of digit-like parameter values
+    if value.isdigit():
+        kb.cache.intBoundaries = kb.cache.intBoundaries or sorted(copy.deepcopy(conf.boundaries), key=lambda boundary: any(_ in (boundary.prefix or "") or _ in (boundary.suffix or "") for _ in ('"', '\'')))
+        boundaries = kb.cache.intBoundaries
+    else:
+        boundaries = conf.boundaries
+
     # Set the flag for SQL injection test mode
     kb.testMode = True
 
@@ -89,6 +112,9 @@ def checkSqlInjection(place, parameter, value):
     tests = getSortedInjectionTests()
     seenPayload = set()
 
+    kb.data.setdefault("randomInt", str(randomInt(10)))
+    kb.data.setdefault("randomStr", str(randomStr(10)))
+
     while tests:
         test = tests.pop(0)
 
@@ -102,7 +128,7 @@ def checkSqlInjection(place, parameter, value):
                 # then attempt to identify with a simple DBMS specific boolean-based
                 # test what the DBMS may be
                 if not injection.dbms and PAYLOAD.TECHNIQUE.BOOLEAN in injection.data:
-                    if not Backend.getIdentifiedDbms() and kb.heuristicDbms is False:
+                    if not Backend.getIdentifiedDbms() and kb.heuristicDbms is None and not kb.droppingRequests:
                         kb.heuristicDbms = heuristicCheckDbms(injection)
 
                 # If the DBMS has already been fingerprinted (via DBMS-specific
@@ -113,7 +139,7 @@ def checkSqlInjection(place, parameter, value):
                    SUPPORTED_DBMS, True) or kb.heuristicDbms or injection.dbms):
                     msg = "it looks like the back-end DBMS is '%s'. " % (Format.getErrorParsedDBMSes() or kb.heuristicDbms or injection.dbms)
                     msg += "Do you want to skip test payloads specific for other DBMSes? [Y/n]"
-                    kb.reduceTests = (Backend.getErrorParsedDBMSes() or [kb.heuristicDbms]) if readInput(msg, default='Y').upper() == 'Y' else []
+                    kb.reduceTests = (Backend.getErrorParsedDBMSes() or [kb.heuristicDbms]) if readInput(msg, default='Y', boolean=True) else []
 
             # If the DBMS has been fingerprinted (via DBMS-specific error
             # message, via simple heuristic check or via DBMS-specific
@@ -128,12 +154,13 @@ def checkSqlInjection(place, parameter, value):
                 msg += " and " if conf.level < 5 and conf.risk < 3 else ""
                 msg += "risk (%d)" % conf.risk if conf.risk < 3 else ""
                 msg += " values? [Y/n]" if conf.level < 5 and conf.risk < 3 else " value? [Y/n]"
-                kb.extendTests = (Backend.getErrorParsedDBMSes() or [kb.heuristicDbms]) if readInput(msg, default='Y').upper() == 'Y' else []
+                kb.extendTests = (Backend.getErrorParsedDBMSes() or [kb.heuristicDbms]) if readInput(msg, default='Y', boolean=True) else []
 
             title = test.title
             kb.testType = stype = test.stype
             clause = test.clause
             unionExtended = False
+            trueCode, falseCode = None, None
 
             if stype == PAYLOAD.TECHNIQUE.UNION:
                 configUnion(test.request.char)
@@ -165,17 +192,18 @@ def checkSqlInjection(place, parameter, value):
                     lower, upper = int(match.group(1)), int(match.group(2))
                     for _ in (lower, upper):
                         if _ > 1:
+                            __ = 2 * (_ - 1) + 1 if _ == lower else 2 * _
                             unionExtended = True
-                            test.request.columns = re.sub(r"\b%d\b" % _, str(2 * _), test.request.columns)
+                            test.request.columns = re.sub(r"\b%d\b" % _, str(__), test.request.columns)
-                            title = re.sub(r"\b%d\b" % _, str(2 * _), title)
+                            title = re.sub(r"\b%d\b" % _, str(__), title)
-                            test.title = re.sub(r"\b%d\b" % _, str(2 * _), test.title)
+                            test.title = re.sub(r"\b%d\b" % _, str(__), test.title)
 
             # Skip test if the user's wants to test only for a specific
             # technique
             if conf.tech and isinstance(conf.tech, list) and stype not in conf.tech:
                 debugMsg = "skipping test '%s' because the user " % title
                 debugMsg += "specified to test only for "
-                debugMsg += "%s techniques" % " & ".join(map(lambda x: PAYLOAD.SQLINJECTION[x], conf.tech))
+                debugMsg += "%s techniques" % " & ".join(PAYLOAD.SQLINJECTION[_] for _ in conf.tech)
                 logger.debug(debugMsg)
                 continue
 
@@ -204,6 +232,16 @@ def checkSqlInjection(place, parameter, value):
                     logger.debug(debugMsg)
                     continue
 
+            # Skip tests if title, vector or DBMS is included by the
+            # given skip filter
+            if conf.testSkip and any(conf.testSkip in str(item) or \
+               re.search(conf.testSkip, str(item), re.I) for item in \
+               (test.title, test.vector, payloadDbms)):
+                    debugMsg = "skipping test '%s' because its " % title
+                    debugMsg += "name/vector/DBMS is included by the given skip filter"
+                    logger.debug(debugMsg)
+                    continue
+
             if payloadDbms is not None:
                 # Skip DBMS-specific test if it does not match the user's
                 # provided DBMS
@@ -285,12 +323,6 @@ def checkSqlInjection(place, parameter, value):
             comment = agent.getComment(test.request) if len(conf.boundaries) > 1 else None
             fstPayload = agent.cleanupPayload(test.request.payload, origValue=value if place not in (PLACE.URI, PLACE.CUSTOM_POST, PLACE.CUSTOM_HEADER) else None)
 
-            # Favoring non-string specific boundaries in case of digit-like parameter values
-            if value.isdigit():
-                boundaries = sorted(copy.deepcopy(conf.boundaries), key=lambda x: any(_ in (x.prefix or "") or _ in (x.suffix or "") for _ in ('"', '\'')))
-            else:
-                boundaries = conf.boundaries
-
             for boundary in boundaries:
                 injectable = False
 
@@ -362,8 +394,6 @@ def checkSqlInjection(place, parameter, value):
                         # Use different page template than the original
                         # one as we are changing parameters value, which
                         # will likely result in a different content
-                        kb.data.setdefault("randomInt", str(randomInt(10)))
-                        kb.data.setdefault("randomStr", str(randomStr(10)))
 
                         if conf.invalidLogical:
                             _ = int(kb.data.randomInt[:2])
@@ -423,47 +453,110 @@ def checkSqlInjection(place, parameter, value):
                             kb.matchRatio = None
                             kb.negativeLogic = (where == PAYLOAD.WHERE.NEGATIVE)
                             Request.queryPage(genCmpPayload(), place, raise404=False)
-                            falsePage = threadData.lastComparisonPage or ""
+                            falsePage, falseHeaders, falseCode = threadData.lastComparisonPage or "", threadData.lastComparisonHeaders, threadData.lastComparisonCode
+                            falseRawResponse = "%s%s" % (falseHeaders, falsePage)
 
                             # Perform the test's True request
                             trueResult = Request.queryPage(reqPayload, place, raise404=False)
-                            truePage = threadData.lastComparisonPage or ""
+                            truePage, trueHeaders, trueCode = threadData.lastComparisonPage or "", threadData.lastComparisonHeaders, threadData.lastComparisonCode
+                            trueRawResponse = "%s%s" % (trueHeaders, truePage)
 
                             if trueResult and not(truePage == falsePage and not kb.nullConnection):
+                                # Perform the test's False request
                                 falseResult = Request.queryPage(genCmpPayload(), place, raise404=False)
 
-                                # Perform the test's False request
                                 if not falseResult:
-                                    infoMsg = "%s parameter '%s' seems to be '%s' injectable " % (paramType, parameter, title)
+                                    if kb.negativeLogic:
-                                    logger.info(infoMsg)
+                                        boundPayload = agent.prefixQuery(kb.data.randomStr, prefix, where, clause)
+                                        boundPayload = agent.suffixQuery(boundPayload, comment, suffix, where)
+                                        errorPayload = agent.payload(place, parameter, newValue=boundPayload, where=where)
+
+                                        errorResult = Request.queryPage(errorPayload, place, raise404=False)
+                                        if errorResult:
+                                            continue
+                                    elif not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
+                                        _ = comparison(kb.heuristicPage, None, getRatioValue=True)
+                                        if _ > kb.matchRatio:
+                                            kb.matchRatio = _
+                                            logger.debug("adjusting match ratio for current parameter to %.3f" % kb.matchRatio)
 
                                     injectable = True
 
-                            if not injectable and not any((conf.string, conf.notString, conf.regexp)) and kb.pageStable:
+                                elif threadData.lastComparisonRatio > UPPER_RATIO_BOUND and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
-                                trueSet = set(extractTextTagContent(truePage))
+                                    originalSet = set(getFilteredPageContent(kb.pageTemplate, True, "\n").split("\n"))
-                                falseSet = set(extractTextTagContent(falsePage))
+                                    trueSet = set(getFilteredPageContent(truePage, True, "\n").split("\n"))
-                                candidates = filter(None, (_.strip() if _.strip() in (kb.pageTemplate or "") and _.strip() not in falsePage and _.strip() not in threadData.lastComparisonHeaders else None for _ in (trueSet - falseSet)))
+                                    falseSet = set(getFilteredPageContent(falsePage, True, "\n").split("\n"))
 
-                                if candidates:
+                                    if originalSet == trueSet != falseSet:
-                                    conf.string = candidates[0]
+                                        candidates = trueSet - falseSet
-                                    infoMsg = "%s parameter '%s' seems to be '%s' injectable (with --string=\"%s\")" % (paramType, parameter, title, repr(conf.string).lstrip('u').strip("'"))
-                                    logger.info(infoMsg)
 
-                                    injectable = True
+                                        if candidates:
+                                            candidates = sorted(candidates, key=lambda _: len(_))
+                                            for candidate in candidates:
+                                                if re.match(r"\A[\w.,! ]+\Z", candidate) and ' ' in candidate and candidate.strip() and len(candidate) > CANDIDATE_SENTENCE_MIN_LENGTH:
+                                                    conf.string = candidate
+                                                    injectable = True
+
+                                                    infoMsg = "%s parameter '%s' appears to be '%s' injectable (with --string=\"%s\")" % (paramType, parameter, title, repr(conf.string).lstrip('u').strip("'"))
+                                                    logger.info(infoMsg)
+
+                                                    break
+
+                            if injectable:
+                                if kb.pageStable and not any((conf.string, conf.notString, conf.regexp, conf.code, kb.nullConnection)):
+                                    if all((falseCode, trueCode)) and falseCode != trueCode:
+                                        conf.code = trueCode
+
+                                        infoMsg = "%s parameter '%s' appears to be '%s' injectable (with --code=%d)" % (paramType, parameter, title, conf.code)
+                                        logger.info(infoMsg)
+                                    else:
+                                        trueSet = set(extractTextTagContent(trueRawResponse))
+                                        trueSet = trueSet.union(__ for _ in trueSet for __ in _.split())
+
+                                        falseSet = set(extractTextTagContent(falseRawResponse))
+                                        falseSet = falseSet.union(__ for _ in falseSet for __ in _.split())
+
+                                        candidates = filter(None, (_.strip() if _.strip() in trueRawResponse and _.strip() not in falseRawResponse else None for _ in (trueSet - falseSet)))
+
+                                        if candidates:
+                                            candidates = sorted(candidates, key=lambda _: len(_))
+                                            for candidate in candidates:
+                                                if re.match(r"\A\w+\Z", candidate):
+                                                    break
+
+                                            conf.string = candidate
+
+                                            infoMsg = "%s parameter '%s' appears to be '%s' injectable (with --string=\"%s\")" % (paramType, parameter, title, repr(conf.string).lstrip('u').strip("'"))
+                                            logger.info(infoMsg)
+
+                                        if not any((conf.string, conf.notString)):
+                                            candidates = filter(None, (_.strip() if _.strip() in falseRawResponse and _.strip() not in trueRawResponse else None for _ in (falseSet - trueSet)))
+
+                                            if candidates:
+                                                candidates = sorted(candidates, key=lambda _: len(_))
+                                                for candidate in candidates:
+                                                    if re.match(r"\A\w+\Z", candidate):
+                                                        break
+
+                                                conf.notString = candidate
+
+                                                infoMsg = "%s parameter '%s' appears to be '%s' injectable (with --not-string=\"%s\")" % (paramType, parameter, title, repr(conf.notString).lstrip('u').strip("'"))
+                                                logger.info(infoMsg)
+
+                                if not any((conf.string, conf.notString, conf.code)):
+                                    infoMsg = "%s parameter '%s' appears to be '%s' injectable " % (paramType, parameter, title)
+                                    singleTimeLogMessage(infoMsg)
 
                         # In case of error-based SQL injection
                         elif method == PAYLOAD.METHOD.GREP:
                             # Perform the test's request and grep the response
                             # body for the test's <grep> regular expression
                             try:
-                                page, headers = Request.queryPage(reqPayload, place, content=True, raise404=False)
+                                page, headers, _ = Request.queryPage(reqPayload, place, content=True, raise404=False)
                                 output = extractRegexResult(check, page, re.DOTALL | re.IGNORECASE) \
-                                        or extractRegexResult(check, listToStrValue( \
+                                        or extractRegexResult(check, threadData.lastHTTPError[2] if wasLastResponseHTTPError() else None, re.DOTALL | re.IGNORECASE) \
-                                        [headers[key] for key in headers.keys() if key.lower() != URI_HTTP_HEADER.lower()] \
+                                        or extractRegexResult(check, listToStrValue([headers[key] for key in headers.keys() if key.lower() != URI_HTTP_HEADER.lower()] if headers else None), re.DOTALL | re.IGNORECASE) \
-                                        if headers else None), re.DOTALL | re.IGNORECASE) \
+                                        or extractRegexResult(check, threadData.lastRedirectMsg[1] if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == threadData.lastRequestUID else None, re.DOTALL | re.IGNORECASE)
-                                        or extractRegexResult(check, threadData.lastRedirectMsg[1] \
-                                        if threadData.lastRedirectMsg and threadData.lastRedirectMsg[0] == \
-                                        threadData.lastRequestUID else None, re.DOTALL | re.IGNORECASE)
 
                                 if output:
                                     result = output == "1"
@@ -485,13 +578,20 @@ def checkSqlInjection(place, parameter, value):
                         elif method == PAYLOAD.METHOD.TIME:
                             # Perform the test's request
                             trueResult = Request.queryPage(reqPayload, place, timeBasedCompare=True, raise404=False)
+                            trueCode = threadData.lastCode
 
                             if trueResult:
+                                # Extra validation step (e.g. to check for DROP protection mechanisms)
+                                if SLEEP_TIME_MARKER in reqPayload:
+                                    falseResult = Request.queryPage(reqPayload.replace(SLEEP_TIME_MARKER, "0"), place, timeBasedCompare=True, raise404=False)
+                                    if falseResult:
+                                        continue
+
                                 # Confirm test's results
                                 trueResult = Request.queryPage(reqPayload, place, timeBasedCompare=True, raise404=False)
 
                                 if trueResult:
-                                    infoMsg = "%s parameter '%s' seems to be '%s' injectable " % (paramType, parameter, title)
+                                    infoMsg = "%s parameter '%s' appears to be '%s' injectable " % (paramType, parameter, title)
                                     logger.info(infoMsg)
 
                                     injectable = True
@@ -511,7 +611,7 @@ def checkSqlInjection(place, parameter, value):
                                     warnMsg = "using unescaped version of the test "
                                     warnMsg += "because of zero knowledge of the "
                                     warnMsg += "back-end DBMS. You can try to "
-                                    warnMsg += "explicitly set it using option '--dbms'"
+                                    warnMsg += "explicitly set it with option '--dbms'"
                                     singleTimeWarnMessage(warnMsg)
                                 else:
                                     Backend.forceDbms(kb.heuristicDbms)
@@ -530,7 +630,8 @@ def checkSqlInjection(place, parameter, value):
                                         msg += "extended UNION tests if there is not "
                                         msg += "at least one other (potential) "
                                         msg += "technique found. Do you want to skip? [Y/n] "
-                                        kb.futileUnion = readInput(msg, default="Y").strip().upper() == 'N'
+
+                                        kb.futileUnion = not readInput(msg, default='Y', boolean=True)
                                     if kb.futileUnion is False:
                                         continue
 
@@ -571,20 +672,20 @@ def checkSqlInjection(place, parameter, value):
 
                         # Feed with test details every time a test is successful
                         if hasattr(test, "details"):
-                            for dKey, dValue in test.details.items():
+                            for key, value in test.details.items():
-                                if dKey == "dbms":
+                                if key == "dbms":
-                                    injection.dbms = dValue
+                                    injection.dbms = value
 
-                                    if not isinstance(dValue, list):
+                                    if not isinstance(value, list):
-                                        Backend.setDbms(dValue)
+                                        Backend.setDbms(value)
                                     else:
-                                        Backend.forceDbms(dValue[0], True)
+                                        Backend.forceDbms(value[0], True)
 
-                                elif dKey == "dbms_version" and injection.dbms_version is None and not conf.testFilter:
+                                elif key == "dbms_version" and injection.dbms_version is None and not conf.testFilter:
-                                    injection.dbms_version = Backend.setVersion(dValue)
+                                    injection.dbms_version = Backend.setVersion(value)
 
-                                elif dKey == "os" and injection.os is None:
+                                elif key == "os" and injection.os is None:
-                                    injection.os = Backend.setOs(dValue)
+                                    injection.os = Backend.setOs(value)
 
                         if vector is None and "vector" in test and test.vector is not None:
                             vector = test.vector
@@ -597,9 +698,12 @@ def checkSqlInjection(place, parameter, value):
                         injection.data[stype].comment = comment
                         injection.data[stype].templatePayload = templatePayload
                         injection.data[stype].matchRatio = kb.matchRatio
+                        injection.data[stype].trueCode = trueCode
+                        injection.data[stype].falseCode = falseCode
 
                         injection.conf.textOnly = conf.textOnly
                         injection.conf.titles = conf.titles
+                        injection.conf.code = conf.code
                         injection.conf.string = conf.string
                         injection.conf.notString = conf.notString
                         injection.conf.regexp = conf.regexp
@@ -613,7 +717,7 @@ def checkSqlInjection(place, parameter, value):
                                 infoMsg = "executing alerting shell command(s) ('%s')" % conf.alert
                                 logger.info(infoMsg)
 
-                                process = execute(conf.alert, shell=True)
+                                process = subprocess.Popen(conf.alert, shell=True)
                                 process.wait()
 
                             kb.alerted = True
@@ -634,25 +738,23 @@ def checkSqlInjection(place, parameter, value):
             logger.warn(warnMsg)
 
             msg = "how do you want to proceed? [(S)kip current test/(e)nd detection phase/(n)ext parameter/(c)hange verbosity/(q)uit]"
-            choice = readInput(msg, default="S", checkBatch=False)
+            choice = readInput(msg, default='S', checkBatch=False).upper()
 
-            if choice[0] in ("s", "S"):
+            if choice == 'C':
-                pass
-            elif choice[0] in ("c", "C"):
                 choice = None
                 while not ((choice or "").isdigit() and 0 <= int(choice) <= 6):
                     if choice:
                         logger.warn("invalid value")
                     msg = "enter new verbosity level: [0-6] "
-                    choice = readInput(msg, default=str(conf.verbose), checkBatch=False).strip()
+                    choice = readInput(msg, default=str(conf.verbose), checkBatch=False)
                 conf.verbose = int(choice)
                 setVerbosity()
                 tests.insert(0, test)
-            elif choice[0] in ("n", "N"):
+            elif choice == 'N':
                 return None
-            elif choice[0] in ("e", "E"):
+            elif choice == 'E':
                 kb.endDetection = True
-            elif choice[0] in ("q", "Q"):
+            elif choice == 'Q':
                 raise SqlmapUserQuitException
 
         finally:
@@ -664,19 +766,20 @@ def checkSqlInjection(place, parameter, value):
     # Return the injection object
     if injection.place is not None and injection.parameter is not None:
         if not conf.dropSetCookie and PAYLOAD.TECHNIQUE.BOOLEAN in injection.data and injection.data[PAYLOAD.TECHNIQUE.BOOLEAN].vector.startswith('OR'):
-            warnMsg = "in OR boolean-based injections, please consider usage "
+            warnMsg = "in OR boolean-based injection cases, please consider usage "
             warnMsg += "of switch '--drop-set-cookie' if you experience any "
             warnMsg += "problems during data retrieval"
             logger.warn(warnMsg)
 
-        injection = checkFalsePositives(injection)
+        if not checkFalsePositives(injection):
-
-        if not injection:
             kb.vulnHosts.remove(conf.hostname)
+            if NOTE.FALSE_POSITIVE_OR_UNEXPLOITABLE not in injection.notes:
+                injection.notes.append(NOTE.FALSE_POSITIVE_OR_UNEXPLOITABLE)
+
     else:
         injection = None
 
-    if injection:
+    if injection and NOTE.FALSE_POSITIVE_OR_UNEXPLOITABLE not in injection.notes:
         checkSuhosinPatch(injection)
         checkFilteredChars(injection)
 
@@ -695,12 +798,12 @@ def heuristicCheckDbms(injection):
     kb.injection = injection
 
     for dbms in getPublicTypeMembers(DBMS, True):
-        if not FROM_DUMMY_TABLE.get(dbms, ""):
-            continue
-
         randStr1, randStr2 = randomStr(), randomStr()
         Backend.forceDbms(dbms)
 
+        if conf.noEscape and dbms not in FROM_DUMMY_TABLE:
+            continue
+
         if checkBooleanExpression("(SELECT '%s'%s)='%s'" % (randStr1, FROM_DUMMY_TABLE.get(dbms, ""), randStr1)):
             if not checkBooleanExpression("(SELECT '%s'%s)='%s'" % (randStr1, FROM_DUMMY_TABLE.get(dbms, ""), randStr2)):
                 retVal = dbms
@@ -714,6 +817,8 @@ def heuristicCheckDbms(injection):
         infoMsg += "could be '%s' " % retVal
         logger.info(infoMsg)
 
+        kb.heuristicExtendedDbms = retVal
+
     return retVal
 
 def checkFalsePositives(injection):
@@ -721,7 +826,7 @@ def checkFalsePositives(injection):
     Checks for false positives (only in single special cases)
     """
 
-    retVal = injection
+    retVal = True
 
     if all(_ in (PAYLOAD.TECHNIQUE.BOOLEAN, PAYLOAD.TECHNIQUE.TIME, PAYLOAD.TECHNIQUE.STACKED) for _ in injection.data) or\
       (len(injection.data) == 1 and PAYLOAD.TECHNIQUE.UNION in injection.data and "Generic" in injection.data[PAYLOAD.TECHNIQUE.UNION].title):
@@ -747,26 +852,30 @@ def checkFalsePositives(injection):
                     break
 
             if not checkBooleanExpression("%d=%d" % (randInt1, randInt1)):
-                retVal = None
+                retVal = False
                 break
 
             # Just in case if DBMS hasn't properly recovered from previous delayed request
             if PAYLOAD.TECHNIQUE.BOOLEAN not in injection.data:
                 checkBooleanExpression("%d=%d" % (randInt1, randInt2))
 
-            if checkBooleanExpression("%d=%d" % (randInt1, randInt3)):
+            if checkBooleanExpression("%d=%d" % (randInt1, randInt3)):          # this must not be evaluated to True
-                retVal = None
+                retVal = False
                 break
 
-            elif checkBooleanExpression("%d=%d" % (randInt3, randInt2)):
+            elif checkBooleanExpression("%d=%d" % (randInt3, randInt2)):        # this must not be evaluated to True
-                retVal = None
+                retVal = False
                 break
 
-            elif not checkBooleanExpression("%d=%d" % (randInt2, randInt2)):
+            elif not checkBooleanExpression("%d=%d" % (randInt2, randInt2)):    # this must be evaluated to True
-                retVal = None
+                retVal = False
                 break
 
-        if retVal is None:
+            elif checkBooleanExpression("%d %d" % (randInt3, randInt2)):        # this must not be evaluated to True (invalid statement)
+                retVal = False
+                break
+
+        if not retVal:
             warnMsg = "false positive or unexploitable injection point detected"
             logger.warn(warnMsg)
 
@@ -833,8 +942,10 @@ def heuristicCheckSqlInjection(place, parameter):
 
     origValue = conf.paramDict[place][parameter]
     paramType = conf.method if conf.method not in (None, HTTPMETHOD.GET, HTTPMETHOD.POST) else place
+
     prefix = ""
     suffix = ""
+    randStr = ""
 
     if conf.prefix or conf.suffix:
         if conf.prefix:
@@ -843,17 +954,16 @@ def heuristicCheckSqlInjection(place, parameter):
         if conf.suffix:
             suffix = conf.suffix
 
-    randStr = ""
+    while randStr.count('\'') != 1 or randStr.count('\"') != 1:
-
-    while '\'' not in randStr:
         randStr = randomStr(length=10, alphabet=HEURISTIC_CHECK_ALPHABET)
 
     kb.heuristicMode = True
 
     payload = "%s%s%s" % (prefix, randStr, suffix)
     payload = agent.payload(place, parameter, newValue=payload)
-    page, _ = Request.queryPage(payload, place, content=True, raise404=False)
+    page, _, _ = Request.queryPage(payload, place, content=True, raise404=False)
 
+    kb.heuristicPage = page
     kb.heuristicMode = False
 
     parseFilePaths(page)
@@ -875,7 +985,7 @@ def heuristicCheckSqlInjection(place, parameter):
 
         if not result:
             randStr = randomStr()
-            payload = "%s%s%s" % (prefix, "%s%s" % (origValue, randStr), suffix)
+            payload = "%s%s%s" % (prefix, "%s.%d%s" % (origValue, random.randint(1, 9), randStr), suffix)
             payload = agent.payload(place, parameter, newValue=payload, where=PAYLOAD.WHERE.REPLACE)
             casting = Request.queryPage(payload, place, raise404=False)
 
@@ -889,7 +999,7 @@ def heuristicCheckSqlInjection(place, parameter):
 
         if kb.ignoreCasted is None:
             message = "do you want to skip those kind of cases (and save scanning time)? %s " % ("[Y/n]" if conf.multipleTargets else "[y/N]")
-            kb.ignoreCasted = readInput(message, default='Y' if conf.multipleTargets else 'N').upper() != 'N'
+            kb.ignoreCasted = readInput(message, default='Y' if conf.multipleTargets else 'N', boolean=True)
 
     elif result:
         infoMsg += "be injectable"
@@ -903,18 +1013,26 @@ def heuristicCheckSqlInjection(place, parameter):
 
     kb.heuristicMode = True
 
-    value = "%s%s%s" % (randomStr(), DUMMY_XSS_CHECK_APPENDIX, randomStr())
+    randStr1, randStr2 = randomStr(NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH), randomStr(NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH)
+    value = "%s%s%s" % (randStr1, DUMMY_NON_SQLI_CHECK_APPENDIX, randStr2)
     payload = "%s%s%s" % (prefix, "'%s" % value, suffix)
     payload = agent.payload(place, parameter, newValue=payload)
-    page, _ = Request.queryPage(payload, place, content=True, raise404=False)
+    page, _, _ = Request.queryPage(payload, place, content=True, raise404=False)
 
     paramType = conf.method if conf.method not in (None, HTTPMETHOD.GET, HTTPMETHOD.POST) else place
 
-    if value in (page or ""):
+    if value.lower() in (page or "").lower():
         infoMsg = "heuristic (XSS) test shows that %s parameter " % paramType
-        infoMsg += "'%s' might be vulnerable to XSS attacks" % parameter
+        infoMsg += "'%s' might be vulnerable to cross-site scripting attacks" % parameter
         logger.info(infoMsg)
 
+    for match in re.finditer(FI_ERROR_REGEX, page or ""):
+        if randStr1.lower() in match.group(0).lower():
+            infoMsg = "heuristic (FI) test shows that %s parameter " % paramType
+            infoMsg += "'%s' might be vulnerable to file inclusion attacks" % parameter
+            logger.info(infoMsg)
+            break
+
     kb.heuristicMode = False
 
     return kb.heuristicTest
@@ -974,12 +1092,22 @@ def checkDynamicContent(firstPage, secondPage):
         logger.critical(warnMsg)
         return
 
-    seqMatcher = getCurrentThreadData().seqMatcher
+    if firstPage and secondPage and any(len(_) > MAX_DIFFLIB_SEQUENCE_LENGTH for _ in (firstPage, secondPage)):
-    seqMatcher.set_seq1(firstPage)
+        ratio = None
-    seqMatcher.set_seq2(secondPage)
+    else:
+        try:
+            seqMatcher = getCurrentThreadData().seqMatcher
+            seqMatcher.set_seq1(firstPage)
+            seqMatcher.set_seq2(secondPage)
+            ratio = seqMatcher.quick_ratio()
+        except MemoryError:
+            ratio = None
+
+    if ratio is None:
+        kb.skipSeqMatcher = True
 
     # In case of an intolerable difference turn on dynamicity removal engine
-    if seqMatcher.quick_ratio() <= UPPER_RATIO_BOUND:
+    elif ratio <= UPPER_RATIO_BOUND:
         findDynamicContent(firstPage, secondPage)
 
         count = 0
@@ -998,7 +1126,7 @@ def checkDynamicContent(firstPage, secondPage):
             warnMsg += ". sqlmap is going to retry the request"
             logger.critical(warnMsg)
 
-            secondPage, _ = Request.queryPage(content=True)
+            secondPage, _, _ = Request.queryPage(content=True)
             findDynamicContent(firstPage, secondPage)
 
 def checkStability():
@@ -1021,7 +1149,7 @@ def checkStability():
     delay = max(0, min(1, delay))
     time.sleep(delay)
 
-    secondPage, _ = Request.queryPage(content=True, raise404=False)
+    secondPage, _, _ = Request.queryPage(content=True, noteResponseTime=False, raise404=False)
 
     if kb.redirectChoice:
         return None
@@ -1049,19 +1177,19 @@ def checkStability():
         logger.warn(warnMsg)
 
         message = "how do you want to proceed? [(C)ontinue/(s)tring/(r)egex/(q)uit] "
-        test = readInput(message, default="C")
+        choice = readInput(message, default='C').upper()
 
-        if test and test[0] in ("q", "Q"):
+        if choice == 'Q':
             raise SqlmapUserQuitException
 
-        elif test and test[0] in ("s", "S"):
+        elif choice == 'S':
             showStaticWords(firstPage, secondPage)
 
             message = "please enter value for parameter 'string': "
-            test = readInput(message)
+            string = readInput(message)
 
-            if test:
+            if string:
-                conf.string = test
+                conf.string = string
 
                 if kb.nullConnection:
                     debugMsg = "turning off NULL connection "
@@ -1073,12 +1201,12 @@ def checkStability():
                 errMsg = "Empty value supplied"
                 raise SqlmapNoneDataException(errMsg)
 
-        elif test and test[0] in ("r", "R"):
+        elif choice == 'R':
             message = "please enter value for parameter 'regex': "
-            test = readInput(message)
+            regex = readInput(message)
 
-            if test:
+            if regex:
-                conf.regex = test
+                conf.regex = regex
 
                 if kb.nullConnection:
                     debugMsg = "turning off NULL connection "
@@ -1103,7 +1231,7 @@ def checkString():
     infoMsg += "target URL page content"
     logger.info(infoMsg)
 
-    page, headers = Request.queryPage(content=True)
+    page, headers, _ = Request.queryPage(content=True)
     rawResponse = "%s%s" % (listToStrValue(headers.headers if headers else ""), page)
 
     if conf.string not in rawResponse:
@@ -1122,7 +1250,7 @@ def checkRegexp():
     infoMsg += "the target URL page content"
     logger.info(infoMsg)
 
-    page, headers = Request.queryPage(content=True)
+    page, headers, _ = Request.queryPage(content=True)
     rawResponse = "%s%s" % (listToStrValue(headers.headers if headers else ""), page)
 
     if not re.search(conf.regexp, rawResponse, re.I | re.M):
@@ -1139,12 +1267,20 @@ def checkWaf():
     Reference: http://seclists.org/nmap-dev/2011/q2/att-1005/http-waf-detect.nse
     """
 
-    if any((conf.string, conf.notString, conf.regexp, conf.dummy, conf.offline)):
+    if any((conf.string, conf.notString, conf.regexp, conf.dummy, conf.offline, conf.skipWaf)):
         return None
 
-    dbmMsg = "heuristically checking if the target is protected by "
+    _ = hashDBRetrieve(HASHDB_KEYS.CHECK_WAF_RESULT, True)
-    dbmMsg += "some kind of WAF/IPS/IDS"
+    if _ is not None:
-    logger.debug(dbmMsg)
+        if _:
+            warnMsg = "previous heuristics detected that the target "
+            warnMsg += "is protected by some kind of WAF/IPS/IDS"
+            logger.critical(warnMsg)
+        return _
+
+    infoMsg = "checking if the target is protected by "
+    infoMsg += "some kind of WAF/IPS/IDS"
+    logger.info(infoMsg)
 
     retVal = False
     payload = "%d %s" % (randomInt(), IDS_WAF_CHECK_PAYLOAD)
@@ -1152,12 +1288,16 @@ def checkWaf():
     value = "" if not conf.parameters.get(PLACE.GET) else conf.parameters[PLACE.GET] + DEFAULT_GET_POST_DELIMITER
     value += agent.addPayloadDelimiters("%s=%s" % (randomStr(), payload))
 
+    pushValue(conf.timeout)
+    conf.timeout = IDS_WAF_CHECK_TIMEOUT
+
     try:
         retVal = Request.queryPage(place=PLACE.GET, value=value, getRatioValue=True, noteResponseTime=False, silent=True)[1] < IDS_WAF_CHECK_RATIO
     except SqlmapConnectionException:
         retVal = True
     finally:
         kb.matchRatio = None
+        conf.timeout = popValue()
 
     if retVal:
         warnMsg = "heuristics detected that the target "
@@ -1167,17 +1307,25 @@ def checkWaf():
         if not conf.identifyWaf:
             message = "do you want sqlmap to try to detect backend "
             message += "WAF/IPS/IDS? [y/N] "
-            output = readInput(message, default="N")
 
-            if output and output[0] in ("Y", "y"):
+            if readInput(message, default='N', boolean=True):
                 conf.identifyWaf = True
 
+        if conf.timeout == defaults.timeout:
+            logger.warning("dropping timeout to %d seconds (i.e. '--timeout=%d')" % (IDS_WAF_CHECK_TIMEOUT, IDS_WAF_CHECK_TIMEOUT))
+            conf.timeout = IDS_WAF_CHECK_TIMEOUT
+
+    hashDBWrite(HASHDB_KEYS.CHECK_WAF_RESULT, retVal, True)
+
     return retVal
 
 def identifyWaf():
     if not conf.identifyWaf:
         return None
 
+    if not kb.wafFunctions:
+        setWafFunctions()
+
     kb.testMode = True
 
     infoMsg = "using WAF scripts to detect "
@@ -1201,36 +1349,38 @@ def identifyWaf():
             kb.redirectChoice = popValue()
         return page or "", headers or {}, code
 
-    retVal = False
+    retVal = []
 
     for function, product in kb.wafFunctions:
         try:
-            logger.debug("checking for WAF/IDS/IPS product '%s'" % product)
+            logger.debug("checking for WAF/IPS/IDS product '%s'" % product)
             found = function(_)
         except Exception, ex:
             errMsg = "exception occurred while running "
-            errMsg += "WAF script for '%s' ('%s')" % (product, ex)
+            errMsg += "WAF script for '%s' ('%s')" % (product, getSafeExString(ex))
             logger.critical(errMsg)
 
             found = False
 
         if found:
-            retVal = product
+            errMsg = "WAF/IPS/IDS identified as '%s'" % product
-            break
+            logger.critical(errMsg)
+
+            retVal.append(product)
 
     if retVal:
-        errMsg = "WAF/IDS/IPS identified '%s'. Please " % retVal
-        errMsg += "consider usage of tamper scripts (option '--tamper')"
-        logger.critical(errMsg)
-
         message = "are you sure that you want to "
         message += "continue with further target testing? [y/N] "
-        output = readInput(message, default="N")
+        choice = readInput(message, default='N', boolean=True)
 
-        if output and output[0] not in ("Y", "y"):
+        if not conf.tamper:
+            warnMsg = "please consider usage of tamper scripts (option '--tamper')"
+            singleTimeWarnMessage(warnMsg)
+
+        if not choice:
             raise SqlmapUserQuitException
     else:
-        warnMsg = "no WAF/IDS/IPS product has been identified"
+        warnMsg = "WAF/IPS/IDS product hasn't been identified"
         logger.warn(warnMsg)
 
     kb.testType = None
@@ -1258,7 +1408,7 @@ def checkNullConnection():
         if not page and HTTP_HEADER.CONTENT_LENGTH in (headers or {}):
             kb.nullConnection = NULLCONNECTION.HEAD
 
-            infoMsg = "NULL connection is supported with HEAD header"
+            infoMsg = "NULL connection is supported with HEAD method (Content-Length)"
             logger.info(infoMsg)
         else:
             page, headers, _ = Request.getPage(auxHeaders={HTTP_HEADER.RANGE: "bytes=-1"})
@@ -1266,7 +1416,7 @@ def checkNullConnection():
             if page and len(page) == 1 and HTTP_HEADER.CONTENT_RANGE in (headers or {}):
                 kb.nullConnection = NULLCONNECTION.RANGE
 
-                infoMsg = "NULL connection is supported with GET header "
+                infoMsg = "NULL connection is supported with GET method (Range)"
                 infoMsg += "'%s'" % kb.nullConnection
                 logger.info(infoMsg)
             else:
@@ -1278,8 +1428,8 @@ def checkNullConnection():
                     infoMsg = "NULL connection is supported with 'skip-read' method"
                     logger.info(infoMsg)
 
-    except SqlmapConnectionException, errMsg:
+    except SqlmapConnectionException, ex:
-        errMsg = getUnicode(errMsg)
+        errMsg = getSafeExString(ex)
         raise SqlmapConnectionException(errMsg)
 
     finally:
@@ -1298,7 +1448,7 @@ def checkConnection(suppressOutput=False):
             raise SqlmapConnectionException(errMsg)
         except socket.error, ex:
             errMsg = "problem occurred while "
-            errMsg += "resolving a host name '%s' ('%s')" % (conf.hostname, ex.message)
+            errMsg += "resolving a host name '%s' ('%s')" % (conf.hostname, getSafeExString(ex))
             raise SqlmapConnectionException(errMsg)
 
     if not suppressOutput and not conf.dummy and not conf.offline:
@@ -1307,7 +1457,7 @@ def checkConnection(suppressOutput=False):
 
     try:
         kb.originalPageTime = time.time()
-        page, _ = Request.queryPage(content=True, noteResponseTime=False)
+        page, headers, _ = Request.queryPage(content=True, noteResponseTime=False)
         kb.originalPage = kb.pageTemplate = page
 
         kb.errorIsNone = False
@@ -1326,7 +1476,7 @@ def checkConnection(suppressOutput=False):
         else:
             kb.errorIsNone = True
 
-    except SqlmapConnectionException, errMsg:
+    except SqlmapConnectionException, ex:
         if conf.ipv6:
             warnMsg = "check connection to a provided "
             warnMsg += "IPv6 address with a tool like ping6 "
@@ -1336,14 +1486,14 @@ def checkConnection(suppressOutput=False):
             singleTimeWarnMessage(warnMsg)
 
         if any(code in kb.httpErrorCodes for code in (httplib.NOT_FOUND, )):
-            errMsg = getUnicode(errMsg)
+            errMsg = getSafeExString(ex)
             logger.critical(errMsg)
 
             if conf.multipleTargets:
                 return False
 
             msg = "it is not recommended to continue in this kind of cases. Do you want to quit and make sure that everything is set up properly? [Y/n] "
-            if readInput(msg, default="Y") not in ("n", "N"):
+            if readInput(msg, default='Y', boolean=True):
                 raise SqlmapSilentQuitException
             else:
                 kb.ignoreNotFound = True
@@ -1352,5 +1502,12 @@ def checkConnection(suppressOutput=False):
 
     return True
 
+def checkInternet():
+    content = Request.getPage(url=CHECK_INTERNET_ADDRESS, checking=True)[0]
+    return CHECK_INTERNET_VALUE in (content or "")
+
 def setVerbosity():  # Cross-linked function
     raise NotImplementedError
+
+def setWafFunctions():  # Cross-linked function
+    raise NotImplementedError

lib/controller/controller.py

@@ -1,12 +1,13 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
 import os
 import re
+import time
 
 from lib.controller.action import action
 from lib.controller.checks import checkSqlInjection
@@ -15,6 +16,7 @@ from lib.controller.checks import checkStability
 from lib.controller.checks import checkString
 from lib.controller.checks import checkRegexp
 from lib.controller.checks import checkConnection
+from lib.controller.checks import checkInternet
 from lib.controller.checks import checkNullConnection
 from lib.controller.checks import checkWaf
 from lib.controller.checks import heuristicCheckSqlInjection
@@ -24,7 +26,7 @@ from lib.core.common import dataToStdout
 from lib.core.common import extractRegexResult
 from lib.core.common import getFilteredPageContent
 from lib.core.common import getPublicTypeMembers
-from lib.core.common import getUnicode
+from lib.core.common import getSafeExString
 from lib.core.common import hashDBRetrieve
 from lib.core.common import hashDBWrite
 from lib.core.common import intersect
@@ -45,6 +47,7 @@ from lib.core.enums import CONTENT_TYPE
 from lib.core.enums import HASHDB_KEYS
 from lib.core.enums import HEURISTIC_TEST
 from lib.core.enums import HTTPMETHOD
+from lib.core.enums import NOTE
 from lib.core.enums import PAYLOAD
 from lib.core.enums import PLACE
 from lib.core.exception import SqlmapBaseException
@@ -64,7 +67,6 @@ from lib.core.settings import REFERER_ALIASES
 from lib.core.settings import USER_AGENT_ALIASES
 from lib.core.target import initTargetEnv
 from lib.core.target import setupTargetEnv
-from thirdparty.pagerank.pagerank import get_pagerank
 
 def _selectInjection():
     """
@@ -116,11 +118,11 @@ def _selectInjection():
                 message += "\n"
 
         message += "[q] Quit"
-        select = readInput(message, default="0")
+        choice = readInput(message, default='0').upper()
 
-        if select.isdigit() and int(select) < len(kb.injections) and int(select) >= 0:
+        if choice.isdigit() and int(choice) < len(kb.injections) and int(choice) >= 0:
-            index = int(select)
+            index = int(choice)
-        elif select[0] in ("Q", "q"):
+        elif choice == 'Q':
             raise SqlmapUserQuitException
         else:
             errMsg = "invalid choice"
@@ -140,7 +142,7 @@ def _formatInjection(inj):
         if inj.place == PLACE.CUSTOM_HEADER:
             payload = payload.split(',', 1)[1]
         if stype == PAYLOAD.TECHNIQUE.UNION:
-            count = re.sub(r"(?i)(\(.+\))|(\blimit[^A-Za-z]+)", "", sdata.payload).count(',') + 1
+            count = re.sub(r"(?i)(\(.+\))|(\blimit[^a-z]+)", "", sdata.payload).count(',') + 1
             title = re.sub(r"\d+ to \d+", str(count), title)
             vector = agent.forgeUnionQuery("[QUERY]", vector[0], vector[1], vector[2], None, None, vector[5], vector[6])
             if count == 1:
@@ -161,10 +163,11 @@ def _showInjections():
     else:
         header = "sqlmap resumed the following injection point(s) from stored session"
 
-    if hasattr(conf, "api"):
+    if conf.api:
+        conf.dumper.string("", {"url": conf.url, "query": conf.parameters.get(PLACE.GET), "data": conf.parameters.get(PLACE.POST)}, content_type=CONTENT_TYPE.TARGET)
         conf.dumper.string("", kb.injections, content_type=CONTENT_TYPE.TECHNIQUES)
     else:
-        data = "".join(set(map(lambda x: _formatInjection(x), kb.injections))).rstrip("\n")
+        data = "".join(set(_formatInjection(_) for _ in kb.injections)).rstrip("\n")
         conf.dumper.string(header, data)
 
     if conf.tamper:
@@ -182,8 +185,8 @@ def _randomFillBlankFields(value):
 
     if extractRegexResult(EMPTY_FORM_FIELDS_REGEX, value):
         message = "do you want to fill blank fields with random values? [Y/n] "
-        test = readInput(message, default="Y")
+
-        if not test or test[0] in ("y", "Y"):
+        if readInput(message, default='Y', boolean=True):
             for match in re.finditer(EMPTY_FORM_FIELDS_REGEX, retVal):
                 item = match.group("result")
                 if not any(_ in item for _ in IGNORE_PARAMETERS) and not re.search(ASP_NET_CONTROL_REGEX, item):
@@ -209,9 +212,8 @@ def _saveToHashDB():
             _[key].data.update(injection.data)
     hashDBWrite(HASHDB_KEYS.KB_INJECTIONS, _.values(), True)
 
-    _ = hashDBRetrieve(HASHDB_KEYS.KB_ABS_FILE_PATHS, True) or set()
+    _ = hashDBRetrieve(HASHDB_KEYS.KB_ABS_FILE_PATHS, True)
-    _.update(kb.absFilePaths)
+    hashDBWrite(HASHDB_KEYS.KB_ABS_FILE_PATHS, kb.absFilePaths | (_ if isinstance(_, set) else set()), True)
-    hashDBWrite(HASHDB_KEYS.KB_ABS_FILE_PATHS, _, True)
 
     if not hashDBRetrieve(HASHDB_KEYS.KB_CHARS):
         hashDBWrite(HASHDB_KEYS.KB_CHARS, kb.chars, True)
@@ -224,25 +226,25 @@ def _saveToResultsFile():
         return
 
     results = {}
-    techniques = dict(map(lambda x: (x[1], x[0]), getPublicTypeMembers(PAYLOAD.TECHNIQUE)))
+    techniques = dict((_[1], _[0]) for _ in getPublicTypeMembers(PAYLOAD.TECHNIQUE))
 
-    for inj in kb.injections:
+    for injection in kb.injections + kb.falsePositives:
-        if inj.place is None or inj.parameter is None:
+        if injection.place is None or injection.parameter is None:
             continue
 
-        key = (inj.place, inj.parameter)
+        key = (injection.place, injection.parameter, ';'.join(injection.notes))
         if key not in results:
             results[key] = []
 
-        results[key].extend(inj.data.keys())
+        results[key].extend(injection.data.keys())
 
     for key, value in results.items():
-        place, parameter = key
+        place, parameter, notes = key
-        line = "%s,%s,%s,%s%s" % (safeCSValue(kb.originalUrls.get(conf.url) or conf.url), place, parameter, "".join(map(lambda x: techniques[x][0].upper(), sorted(value))), os.linesep)
+        line = "%s,%s,%s,%s,%s%s" % (safeCSValue(kb.originalUrls.get(conf.url) or conf.url), place, parameter, "".join(techniques[_][0].upper() for _ in sorted(value)), notes, os.linesep)
         conf.resultsFP.writelines(line)
 
     if not results:
-        line = "%s,,,%s" % (conf.url, os.linesep)
+        line = "%s,,,,%s" % (conf.url, os.linesep)
         conf.resultsFP.writelines(line)
 
 def start():
@@ -276,6 +278,21 @@ def start():
 
     for targetUrl, targetMethod, targetData, targetCookie, targetHeaders in kb.targets:
         try:
+
+            if conf.checkInternet:
+                infoMsg = "[INFO] checking for Internet connection"
+                logger.info(infoMsg)
+
+                if not checkInternet():
+                    warnMsg = "[%s] [WARNING] no connection detected" % time.strftime("%X")
+                    dataToStdout(warnMsg)
+
+                    while not checkInternet():
+                        dataToStdout('.')
+                        time.sleep(5)
+
+                    dataToStdout("\n")
+
             conf.url = targetUrl
             conf.method = targetMethod.upper() if targetMethod else targetMethod
             conf.data = targetData
@@ -305,7 +322,9 @@ def start():
                     message = "SQL injection vulnerability has already been detected "
                     message += "against '%s'. Do you want to skip " % conf.hostname
                     message += "further tests involving it? [Y/n]"
-                    kb.skipVulnHost = readInput(message, default="Y").upper() != 'N'
+
+                    kb.skipVulnHost = readInput(message, default='Y', boolean=True)
+
                 testSqlInj = not kb.skipVulnHost
 
             if not testSqlInj:
@@ -316,10 +335,10 @@ def start():
             if conf.multipleTargets:
                 hostCount += 1
 
-                if conf.forms:
+                if conf.forms and conf.method:
-                    message = "[#%d] form:\n%s %s" % (hostCount, conf.method or HTTPMETHOD.GET, targetUrl)
+                    message = "[#%d] form:\n%s %s" % (hostCount, conf.method, targetUrl)
                 else:
-                    message = "URL %d:\n%s %s%s" % (hostCount, HTTPMETHOD.GET, targetUrl, " (PageRank: %s)" % get_pagerank(targetUrl) if conf.googleDork and conf.pageRank else "")
+                    message = "URL %d:\n%s %s" % (hostCount, HTTPMETHOD.GET, targetUrl)
 
                 if conf.cookie:
                     message += "\nCookie: %s" % conf.cookie
@@ -327,14 +346,18 @@ def start():
                 if conf.data is not None:
                     message += "\n%s data: %s" % ((conf.method if conf.method != HTTPMETHOD.GET else conf.method) or HTTPMETHOD.POST, urlencode(conf.data) if conf.data else "")
 
-                if conf.forms:
+                if conf.forms and conf.method:
                     if conf.method == HTTPMETHOD.GET and targetUrl.find("?") == -1:
                         continue
 
                     message += "\ndo you want to test this form? [Y/n/q] "
-                    test = readInput(message, default="Y")
+                    choice = readInput(message, default='Y').upper()
 
-                    if not test or test[0] in ("y", "Y"):
+                    if choice == 'N':
+                        continue
+                    elif choice == 'Q':
+                        break
+                    else:
                         if conf.method != HTTPMETHOD.GET:
                             message = "Edit %s data [default: %s]%s: " % (conf.method, urlencode(conf.data) if conf.data else "None", " (Warning: blank fields detected)" if conf.data and extractRegexResult(EMPTY_FORM_FIELDS_REGEX, conf.data) else "")
                             conf.data = readInput(message, default=conf.data)
@@ -352,21 +375,14 @@ def start():
 
                         parseTargetUrl()
 
-                    elif test[0] in ("n", "N"):
-                        continue
-                    elif test[0] in ("q", "Q"):
-                        break
-
                 else:
                     message += "\ndo you want to test this URL? [Y/n/q]"
-                    test = readInput(message, default="Y")
+                    choice = readInput(message, default='Y').upper()
 
-                    if not test or test[0] in ("y", "Y"):
+                    if choice == 'N':
-                        pass
-                    elif test[0] in ("n", "N"):
                         dataToStdout(os.linesep)
                         continue
-                    elif test[0] in ("q", "Q"):
+                    elif choice == 'Q':
                         break
 
                     infoMsg = "testing URL '%s'" % targetUrl
@@ -421,6 +437,7 @@ def start():
                     skip |= (place == PLACE.USER_AGENT and intersect(USER_AGENT_ALIASES, conf.skip, True) not in ([], None))
                     skip |= (place == PLACE.REFERER and intersect(REFERER_ALIASES, conf.skip, True) not in ([], None))
                     skip |= (place == PLACE.COOKIE and intersect(PLACE.COOKIE, conf.skip, True) not in ([], None))
+                    skip |= (place == PLACE.HOST and intersect(PLACE.HOST, conf.skip, True) not in ([], None))
 
                     skip &= not (place == PLACE.USER_AGENT and intersect(USER_AGENT_ALIASES, conf.testParameter, True))
                     skip &= not (place == PLACE.REFERER and intersect(REFERER_ALIASES, conf.testParameter, True))
@@ -463,7 +480,13 @@ def start():
                             infoMsg = "skipping randomizing %s parameter '%s'" % (paramType, parameter)
                             logger.info(infoMsg)
 
-                        elif parameter in conf.skip:
+                        elif parameter in conf.skip or kb.postHint and parameter.split(' ')[-1] in conf.skip:
+                            testSqlInj = False
+
+                            infoMsg = "skipping %s parameter '%s'" % (paramType, parameter)
+                            logger.info(infoMsg)
+
+                        elif conf.paramExclude and (re.search(conf.paramExclude, parameter, re.I) or kb.postHint and re.search(conf.paramExclude, parameter.split(' ')[-1], re.I)):
                             testSqlInj = False
 
                             infoMsg = "skipping %s parameter '%s'" % (paramType, parameter)
@@ -486,7 +509,7 @@ def start():
                             check = checkDynParam(place, parameter, value)
 
                             if not check:
-                                warnMsg = "%s parameter '%s' does not appear dynamic" % (paramType, parameter)
+                                warnMsg = "%s parameter '%s' does not appear to be dynamic" % (paramType, parameter)
                                 logger.warn(warnMsg)
 
                                 if conf.skipStatic:
@@ -520,24 +543,30 @@ def start():
 
                                 injection = checkSqlInjection(place, parameter, value)
                                 proceed = not kb.endDetection
+                                injectable = False
 
-                                if injection is not None and injection.place is not None:
+                                if getattr(injection, "place", None) is not None:
-                                    kb.injections.append(injection)
+                                    if NOTE.FALSE_POSITIVE_OR_UNEXPLOITABLE in injection.notes:
+                                        kb.falsePositives.append(injection)
+                                    else:
+                                        injectable = True
 
-                                    # In case when user wants to end detection phase (Ctrl+C)
+                                        kb.injections.append(injection)
-                                    if not proceed:
-                                        break
 
-                                    msg = "%s parameter '%s' " % (injection.place, injection.parameter)
+                                        # In case when user wants to end detection phase (Ctrl+C)
-                                    msg += "is vulnerable. Do you want to keep testing the others (if any)? [y/N] "
+                                        if not proceed:
-                                    test = readInput(msg, default="N")
+                                            break
 
-                                    if test[0] not in ("y", "Y"):
+                                        msg = "%s parameter '%s' " % (injection.place, injection.parameter)
-                                        proceed = False
+                                        msg += "is vulnerable. Do you want to keep testing the others (if any)? [y/N] "
-                                        paramKey = (conf.hostname, conf.path, None, None)
+
-                                        kb.testedParams.add(paramKey)
+                                        if not readInput(msg, default='N', boolean=True):
-                                else:
+                                            proceed = False
-                                    warnMsg = "%s parameter '%s' is not " % (paramType, parameter)
+                                            paramKey = (conf.hostname, conf.path, None, None)
+                                            kb.testedParams.add(paramKey)
+
+                                if not injectable:
+                                    warnMsg = "%s parameter '%s' does not seem to be " % (paramType, parameter)
                                     warnMsg += "injectable"
                                     logger.warn(warnMsg)
 
@@ -585,24 +614,24 @@ def start():
                     if not conf.string and not conf.notString and not conf.regexp:
                         errMsg += " Also, you can try to rerun by providing "
                         errMsg += "either a valid value for option '--string' "
-                        errMsg += "(or '--regexp')"
+                        errMsg += "(or '--regexp')."
                     elif conf.string:
                         errMsg += " Also, you can try to rerun by providing a "
                         errMsg += "valid value for option '--string' as perhaps the string you "
                         errMsg += "have chosen does not match "
-                        errMsg += "exclusively True responses"
+                        errMsg += "exclusively True responses."
                     elif conf.regexp:
                         errMsg += " Also, you can try to rerun by providing a "
                         errMsg += "valid value for option '--regexp' as perhaps the regular "
                         errMsg += "expression that you have chosen "
-                        errMsg += "does not match exclusively True responses"
+                        errMsg += "does not match exclusively True responses."
 
                     if not conf.tamper:
                         errMsg += " If you suspect that there is some kind of protection mechanism "
                         errMsg += "involved (e.g. WAF) maybe you could retry "
                         errMsg += "with an option '--tamper' (e.g. '--tamper=space2comment')"
 
-                    raise SqlmapNotVulnerableException(errMsg)
+                    raise SqlmapNotVulnerableException(errMsg.rstrip('.'))
             else:
                 # Flush the flag
                 kb.testMode = False
@@ -615,9 +644,7 @@ def start():
             if kb.injection.place is not None and kb.injection.parameter is not None:
                 if conf.multipleTargets:
                     message = "do you want to exploit this SQL injection? [Y/n] "
-                    exploit = readInput(message, default="Y")
+                    condition = readInput(message, default='Y', boolean=True)
-
-                    condition = not exploit or exploit[0] in ("y", "Y")
                 else:
                     condition = True
 
@@ -630,13 +657,11 @@ def start():
                 logger.warn(warnMsg)
 
                 message = "do you want to skip to the next target in list? [Y/n/q]"
-                test = readInput(message, default="Y")
+                choice = readInput(message, default='Y').upper()
 
-                if not test or test[0] in ("y", "Y"):
+                if choice == 'N':
-                    pass
-                elif test[0] in ("n", "N"):
                     return False
-                elif test[0] in ("q", "Q"):
+                elif choice == 'Q':
                     raise SqlmapUserQuitException
             else:
                 raise
@@ -648,11 +673,13 @@ def start():
             raise
 
         except SqlmapBaseException, ex:
-            errMsg = getUnicode(ex.message)
+            errMsg = getSafeExString(ex)
 
             if conf.multipleTargets:
+                _saveToResultsFile()
+
                 errMsg += ", skipping to the next %s" % ("form" if conf.forms else "URL")
-                logger.error(errMsg)
+                logger.error(errMsg.lstrip(", "))
             else:
                 logger.critical(errMsg)
                 return False
@@ -669,9 +696,10 @@ def start():
     if kb.dataOutputFlag and not conf.multipleTargets:
         logger.info("fetched data logged to text files under '%s'" % conf.outputPath)
 
-    if conf.multipleTargets and conf.resultsFilename:
+    if conf.multipleTargets:
-        infoMsg = "you can find results of scanning in multiple targets "
+        if conf.resultsFilename:
-        infoMsg += "mode inside the CSV file '%s'" % conf.resultsFilename
+            infoMsg = "you can find results of scanning in multiple targets "
-        logger.info(infoMsg)
+            infoMsg += "mode inside the CSV file '%s'" % conf.resultsFilename
+            logger.info(infoMsg)
 
     return True

lib/controller/handler.py

@@ -1,13 +1,13 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
 from lib.core.common import Backend
 from lib.core.data import conf
-from lib.core.data import logger
+from lib.core.data import kb
 from lib.core.dicts import DBMS_DICT
 from lib.core.enums import DBMS
 from lib.core.settings import MSSQL_ALIASES
@@ -21,6 +21,7 @@ from lib.core.settings import MAXDB_ALIASES
 from lib.core.settings import SYBASE_ALIASES
 from lib.core.settings import DB2_ALIASES
 from lib.core.settings import HSQLDB_ALIASES
+from lib.core.settings import INFORMIX_ALIASES
 from lib.utils.sqlalchemy import SQLAlchemy
 
 from plugins.dbms.mssqlserver import MSSQLServerMap
@@ -45,6 +46,8 @@ from plugins.dbms.db2 import DB2Map
 from plugins.dbms.db2.connector import Connector as DB2Conn
 from plugins.dbms.hsqldb import HSQLDBMap
 from plugins.dbms.hsqldb.connector import Connector as HSQLDBConn
+from plugins.dbms.informix import InformixMap
+from plugins.dbms.informix.connector import Connector as InformixConn
 
 def setHandler():
     """
@@ -64,26 +67,19 @@ def setHandler():
                   (DBMS.SYBASE, SYBASE_ALIASES, SybaseMap, SybaseConn),
                   (DBMS.DB2, DB2_ALIASES, DB2Map, DB2Conn),
                   (DBMS.HSQLDB, HSQLDB_ALIASES, HSQLDBMap, HSQLDBConn),
+                  (DBMS.INFORMIX, INFORMIX_ALIASES, InformixMap, InformixConn),
             ]
 
-    _ = max(_ if (Backend.getIdentifiedDbms() or "").lower() in _[1] else None for _ in items)
+    _ = max(_ if (Backend.getIdentifiedDbms() or kb.heuristicExtendedDbms or "").lower() in _[1] else None for _ in items)
     if _:
         items.remove(_)
         items.insert(0, _)
 
     for dbms, aliases, Handler, Connector in items:
-        if conf.dbms and conf.dbms.lower() != dbms and conf.dbms.lower() not in aliases:
-            debugMsg = "skipping test for %s" % dbms
-            logger.debug(debugMsg)
-            continue
-
         handler = Handler()
         conf.dbmsConnector = Connector()
 
         if conf.direct:
-            logger.debug("forcing timeout to 10 seconds")
-            conf.timeout = 10
-
             dialect = DBMS_DICT[dbms][3]
 
             if dialect:
@@ -101,7 +97,12 @@ def setHandler():
                 conf.dbmsConnector.connect()
 
         if handler.checkDbms():
-            conf.dbmsHandler = handler
+            if kb.resolutionDbms:
+                conf.dbmsHandler = max(_ for _ in items if _[0] == kb.resolutionDbms)[2]()
+            else:
+                conf.dbmsHandler = handler
+
+            conf.dbmsHandler._dbms = dbms
             break
         else:
             conf.dbmsConnector = None

lib/core/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/core/agent.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -17,6 +17,7 @@ from lib.core.common import isTechniqueAvailable
 from lib.core.common import randomInt
 from lib.core.common import randomStr
 from lib.core.common import safeSQLIdentificatorNaming
+from lib.core.common import safeStringFormat
 from lib.core.common import singleTimeWarnMessage
 from lib.core.common import splitFields
 from lib.core.common import unArrayizeValue
@@ -34,12 +35,15 @@ from lib.core.enums import PLACE
 from lib.core.enums import POST_HINT
 from lib.core.exception import SqlmapNoneDataException
 from lib.core.settings import BOUNDARY_BACKSLASH_MARKER
+from lib.core.settings import BOUNDED_INJECTION_MARKER
 from lib.core.settings import CUSTOM_INJECTION_MARK_CHAR
 from lib.core.settings import DEFAULT_COOKIE_DELIMITER
 from lib.core.settings import DEFAULT_GET_POST_DELIMITER
 from lib.core.settings import GENERIC_SQL_COMMENT
+from lib.core.settings import NULL
 from lib.core.settings import PAYLOAD_DELIMITER
 from lib.core.settings import REPLACEMENT_MARKER
+from lib.core.settings import SLEEP_TIME_MARKER
 from lib.core.unescaper import unescaper
 
 class Agent(object):
@@ -59,7 +63,7 @@ class Agent(object):
 
         if Backend.getIdentifiedDbms() in (DBMS.ORACLE,):  # non-standard object(s) make problems to a database connector while returned (e.g. XMLTYPE)
             _, _, _, _, _, _, fieldsToCastStr, _ = self.getFields(query)
-            for field in fieldsToCastStr.split(","):
+            for field in fieldsToCastStr.split(','):
                 query = query.replace(field, self.nullAndCastField(field))
 
         if kb.tamperFunctions:
@@ -94,9 +98,12 @@ class Agent(object):
         paramDict = conf.paramDict[place]
         origValue = getUnicode(paramDict[parameter])
 
-        if place == PLACE.URI:
+        if place == PLACE.URI or BOUNDED_INJECTION_MARKER in origValue:
             paramString = origValue
-            origValue = origValue.split(CUSTOM_INJECTION_MARK_CHAR)[0]
+            if place == PLACE.URI:
+                origValue = origValue.split(CUSTOM_INJECTION_MARK_CHAR)[0]
+            else:
+                origValue = filter(None, (re.search(_, origValue.split(BOUNDED_INJECTION_MARKER)[0]) for _ in (r"\w+\Z", r"[^\"'><]+\Z", r"[^ ]+\Z")))[0].group(0)
             origValue = origValue[origValue.rfind('/') + 1:]
             for char in ('?', '=', ':'):
                 if char in origValue:
@@ -114,7 +121,7 @@ class Agent(object):
         elif place == PLACE.CUSTOM_HEADER:
             paramString = origValue
             origValue = origValue.split(CUSTOM_INJECTION_MARK_CHAR)[0]
-            origValue = origValue[origValue.index(',') + 1:]
+            origValue = origValue[origValue.find(',') + 1:]
             match = re.search(r"([^;]+)=(?P<value>[^;]+);?\Z", origValue)
             if match:
                 origValue = match.group("value")
@@ -124,9 +131,6 @@ class Agent(object):
                 if header.upper() == HTTP_HEADER.AUTHORIZATION.upper():
                     origValue = origValue.split(' ')[-1].split(':')[-1]
 
-        if conf.prefix:
-            value = origValue
-
         if value is None:
             if where == PAYLOAD.WHERE.ORIGINAL:
                 value = origValue
@@ -163,6 +167,9 @@ class Agent(object):
             newValue = newValue.replace(CUSTOM_INJECTION_MARK_CHAR, REPLACEMENT_MARKER)
             retVal = paramString.replace(_, self.addPayloadDelimiters(newValue))
             retVal = retVal.replace(CUSTOM_INJECTION_MARK_CHAR, "").replace(REPLACEMENT_MARKER, CUSTOM_INJECTION_MARK_CHAR)
+        elif BOUNDED_INJECTION_MARKER in paramDict[parameter]:
+            _ = "%s%s" % (origValue, BOUNDED_INJECTION_MARKER)
+            retVal = "%s=%s" % (re.sub(r" (\#\d\*|\(.+\))\Z", "", parameter), paramString.replace(_, self.addPayloadDelimiters(newValue)))
         elif place in (PLACE.USER_AGENT, PLACE.REFERER, PLACE.HOST):
             retVal = paramString.replace(origValue, self.addPayloadDelimiters(newValue))
         else:
@@ -187,12 +194,12 @@ class Agent(object):
 
             if origValue:
                 regex = r"(\A|\b)%s=%s%s" % (re.escape(parameter), re.escape(origValue), r"(\Z|\b)" if origValue[-1].isalnum() else "")
-                retVal = _(regex, "%s=%s" % (parameter, self.addPayloadDelimiters(newValue.replace("\\", "\\\\"))), paramString)
+                retVal = _(regex, "%s=%s" % (parameter, self.addPayloadDelimiters(newValue)), paramString)
             else:
-                retVal = _(r"(\A|\b)%s=%s(\Z|%s|%s|\s)" % (re.escape(parameter), re.escape(origValue), DEFAULT_GET_POST_DELIMITER, DEFAULT_COOKIE_DELIMITER), "%s=%s\g<2>" % (parameter, self.addPayloadDelimiters(newValue.replace("\\", "\\\\"))), paramString)
+                retVal = _(r"(\A|\b)%s=%s(\Z|%s|%s|\s)" % (re.escape(parameter), re.escape(origValue), DEFAULT_GET_POST_DELIMITER, DEFAULT_COOKIE_DELIMITER), "%s=%s\g<2>" % (parameter, self.addPayloadDelimiters(newValue)), paramString)
 
             if retVal == paramString and urlencode(parameter) != parameter:
-                retVal = _(r"(\A|\b)%s=%s" % (re.escape(urlencode(parameter)), re.escape(origValue)), "%s=%s" % (urlencode(parameter), self.addPayloadDelimiters(newValue.replace("\\", "\\\\"))), paramString)
+                retVal = _(r"(\A|\b)%s=%s" % (re.escape(urlencode(parameter)), re.escape(origValue)), "%s=%s" % (urlencode(parameter), self.addPayloadDelimiters(newValue)), paramString)
 
         if retVal:
             retVal = retVal.replace(BOUNDARY_BACKSLASH_MARKER, '\\')
@@ -231,7 +238,7 @@ class Agent(object):
 
         # If we are replacing (<where>) the parameter original value with
         # our payload do not prepend with the prefix
-        if where == PAYLOAD.WHERE.REPLACE and not conf.prefix:
+        if where == PAYLOAD.WHERE.REPLACE:
             query = ""
 
         # If the technique is stacked queries (<stype>) do not put a space
@@ -275,7 +282,7 @@ class Agent(object):
             where = kb.injection.data[kb.technique].where if where is None else where
             comment = kb.injection.data[kb.technique].comment if comment is None else comment
 
-        if Backend.getIdentifiedDbms() == DBMS.ACCESS and comment == GENERIC_SQL_COMMENT:
+        if Backend.getIdentifiedDbms() == DBMS.ACCESS and any((comment or "").startswith(_) for _ in ("--", "[GENERIC_SQL_COMMENT]")):
             comment = queries[DBMS.ACCESS].comment.query
 
         if comment is not None:
@@ -289,7 +296,7 @@ class Agent(object):
         elif suffix and not comment:
             expression += suffix.replace('\\', BOUNDARY_BACKSLASH_MARKER)
 
-        return re.sub(r"(?s);\W*;", ";", expression)
+        return re.sub(r";\W*;", ";", expression)
 
     def cleanupPayload(self, payload, origValue=None):
         if payload is None:
@@ -298,7 +305,7 @@ class Agent(object):
         _ = (
                 ("[DELIMITER_START]", kb.chars.start), ("[DELIMITER_STOP]", kb.chars.stop),\
                 ("[AT_REPLACE]", kb.chars.at), ("[SPACE_REPLACE]", kb.chars.space), ("[DOLLAR_REPLACE]", kb.chars.dollar),\
-                ("[HASH_REPLACE]", kb.chars.hash_),
+                ("[HASH_REPLACE]", kb.chars.hash_), ("[GENERIC_SQL_COMMENT]", GENERIC_SQL_COMMENT)
             )
         payload = reduce(lambda x, y: x.replace(y[0], y[1]), _, payload)
 
@@ -308,8 +315,9 @@ class Agent(object):
         for _ in set(re.findall(r"\[RANDSTR(?:\d+)?\]", payload, re.I)):
             payload = payload.replace(_, randomStr())
 
-        if origValue is not None:
+        if origValue is not None and "[ORIGVALUE]" in payload:
-            payload = payload.replace("[ORIGVALUE]", origValue if origValue.isdigit() else unescaper.escape("'%s'" % origValue))
+            origValue = getUnicode(origValue)
+            payload = getUnicode(payload).replace("[ORIGVALUE]", origValue if origValue.isdigit() else unescaper.escape("'%s'" % origValue))
 
         if "[INFERENCE]" in payload:
             if Backend.getIdentifiedDbms() is not None:
@@ -337,7 +345,7 @@ class Agent(object):
         """
 
         if payload:
-            payload = payload.replace("[SLEEPTIME]", str(conf.timeSec))
+            payload = payload.replace(SLEEP_TIME_MARKER, str(conf.timeSec))
 
         return payload
 
@@ -445,7 +453,7 @@ class Agent(object):
         @rtype: C{str}
         """
 
-        if not Backend.getDbms():
+        if not Backend.getIdentifiedDbms():
             return fields
 
         if fields.startswith("(CASE") or fields.startswith("(IIF") or fields.startswith("SUBSTR") or fields.startswith("MID(") or re.search(r"\A'[^']+'\Z", fields):
@@ -480,7 +488,7 @@ class Agent(object):
         @rtype: C{str}
         """
 
-        prefixRegex = r"(?:\s+(?:FIRST|SKIP)\s+\d+)*"
+        prefixRegex = r"(?:\s+(?:FIRST|SKIP|LIMIT(?: \d+)?)\s+\d+)*"
         fieldsSelectTop = re.search(r"\ASELECT\s+TOP\s+[\d]+\s+(.+?)\s+FROM", query, re.I)
         fieldsSelectRownum = re.search(r"\ASELECT\s+([^()]+?),\s*ROWNUM AS LIMIT FROM", query, re.I)
         fieldsSelectDistinct = re.search(r"\ASELECT%s\s+DISTINCT\((.+?)\)\s+FROM" % prefixRegex, query, re.I)
@@ -496,27 +504,33 @@ class Agent(object):
         if not _:
             fieldsSelectFrom = None
 
+        fieldsToCastStr = fieldsNoSelect
+
         if fieldsSubstr:
             fieldsToCastStr = query
         elif fieldsMinMaxstr:
-            fieldsToCastStr = fieldsMinMaxstr.groups()[0]
+            fieldsToCastStr = fieldsMinMaxstr.group(1)
         elif fieldsExists:
-            fieldsToCastStr = fieldsSelect.groups()[0]
+            if fieldsSelect:
+                fieldsToCastStr = fieldsSelect.group(1)
         elif fieldsSelectTop:
-            fieldsToCastStr = fieldsSelectTop.groups()[0]
+            fieldsToCastStr = fieldsSelectTop.group(1)
         elif fieldsSelectRownum:
-            fieldsToCastStr = fieldsSelectRownum.groups()[0]
+            fieldsToCastStr = fieldsSelectRownum.group(1)
         elif fieldsSelectDistinct:
-            fieldsToCastStr = fieldsSelectDistinct.groups()[0]
+            if Backend.getDbms() in (DBMS.HSQLDB,):
+                fieldsToCastStr = fieldsNoSelect
+            else:
+                fieldsToCastStr = fieldsSelectDistinct.group(1)
         elif fieldsSelectCase:
-            fieldsToCastStr = fieldsSelectCase.groups()[0]
+            fieldsToCastStr = fieldsSelectCase.group(1)
         elif fieldsSelectFrom:
             fieldsToCastStr = query[:unArrayizeValue(_)] if _ else query
             fieldsToCastStr = re.sub(r"\ASELECT%s\s+" % prefixRegex, "", fieldsToCastStr)
         elif fieldsSelect:
-            fieldsToCastStr = fieldsSelect.groups()[0]
+            fieldsToCastStr = fieldsSelect.group(1)
-        else:
+
-            fieldsToCastStr = fieldsNoSelect
+        fieldsToCastStr = fieldsToCastStr or ""
 
         # Function
         if re.search("\A\w+\(.*\)", fieldsToCastStr, re.I) or (fieldsSelectCase and "WHEN use" not in query) or fieldsSubstr:
@@ -584,7 +598,7 @@ class Agent(object):
         else:
             return query
 
-        if Backend.getIdentifiedDbms() in (DBMS.MYSQL,):
+        if Backend.isDbms(DBMS.MYSQL):
             if fieldsExists:
                 concatenatedQuery = concatenatedQuery.replace("SELECT ", "CONCAT('%s'," % kb.chars.start, 1)
                 concatenatedQuery += ",'%s')" % kb.chars.stop
@@ -611,6 +625,7 @@ class Agent(object):
                 concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
                 _ = unArrayizeValue(zeroDepthSearch(concatenatedQuery, " FROM "))
                 concatenatedQuery = "%s||'%s'%s" % (concatenatedQuery[:_], kb.chars.stop, concatenatedQuery[_:])
+                concatenatedQuery = re.sub(r"('%s'\|\|)(.+)(%s)" % (kb.chars.start, re.escape(castedFields)), "\g<2>\g<1>\g<3>", concatenatedQuery)
             elif fieldsSelect:
                 concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
                 concatenatedQuery += "||'%s'" % kb.chars.stop
@@ -656,24 +671,23 @@ class Agent(object):
                 concatenatedQuery = "'%s'&%s&'%s'" % (kb.chars.start, concatenatedQuery, kb.chars.stop)
 
         else:
-            warnMsg = "applying generic concatenation with double pipes ('||')"
+            warnMsg = "applying generic concatenation (CONCAT)"
             singleTimeWarnMessage(warnMsg)
 
             if fieldsExists:
-                concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
+                concatenatedQuery = concatenatedQuery.replace("SELECT ", "CONCAT(CONCAT('%s'," % kb.chars.start, 1)
-                concatenatedQuery += "||'%s'" % kb.chars.stop
+                concatenatedQuery += "),'%s')" % kb.chars.stop
             elif fieldsSelectCase:
-                concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||(SELECT " % kb.chars.start, 1)
+                concatenatedQuery = concatenatedQuery.replace("SELECT ", "CONCAT(CONCAT('%s'," % kb.chars.start, 1)
-                concatenatedQuery += ")||'%s'" % kb.chars.stop
+                concatenatedQuery += "),'%s')" % kb.chars.stop
             elif fieldsSelectFrom:
-                concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
                 _ = unArrayizeValue(zeroDepthSearch(concatenatedQuery, " FROM "))
-                concatenatedQuery = "%s||'%s'%s" % (concatenatedQuery[:_], kb.chars.stop, concatenatedQuery[_:])
+                concatenatedQuery = "%s),'%s')%s" % (concatenatedQuery[:_].replace("SELECT ", "CONCAT(CONCAT('%s'," % kb.chars.start, 1), kb.chars.stop, concatenatedQuery[_:])
             elif fieldsSelect:
-                concatenatedQuery = concatenatedQuery.replace("SELECT ", "'%s'||" % kb.chars.start, 1)
+                concatenatedQuery = concatenatedQuery.replace("SELECT ", "CONCAT(CONCAT('%s'," % kb.chars.start, 1)
-                concatenatedQuery += "||'%s'" % kb.chars.stop
+                concatenatedQuery += "),'%s')" % kb.chars.stop
             elif fieldsNoSelect:
-                concatenatedQuery = "'%s'||%s||'%s'" % (kb.chars.start, concatenatedQuery, kb.chars.stop)
+                concatenatedQuery = "CONCAT(CONCAT('%s',%s),'%s')" % (kb.chars.start, concatenatedQuery, kb.chars.stop)
 
         return concatenatedQuery
 
@@ -710,8 +724,11 @@ class Agent(object):
 
         if conf.uFrom:
             fromTable = " FROM %s" % conf.uFrom
-        else:
+        elif not fromTable:
-            fromTable = fromTable or FROM_DUMMY_TABLE.get(Backend.getIdentifiedDbms(), "")
+            if kb.tableFrom:
+                fromTable = " FROM %s" % kb.tableFrom
+            else:
+                fromTable = FROM_DUMMY_TABLE.get(Backend.getIdentifiedDbms(), "")
 
         if query.startswith("SELECT "):
             query = query[len("SELECT "):]
@@ -744,6 +761,9 @@ class Agent(object):
             intoRegExp = intoRegExp.group(1)
             query = query[:query.index(intoRegExp)]
 
+            position = 0
+            char = NULL
+
         for element in xrange(0, count):
             if element > 0:
                 unionQuery += ','
@@ -838,7 +858,7 @@ class Agent(object):
                     if expression.find(queries[Backend.getIdentifiedDbms()].limitstring.query) > 0:
                         _ = expression.index(queries[Backend.getIdentifiedDbms()].limitstring.query)
                     else:
-                        _ = expression.index("LIMIT ")
+                        _ = re.search(r"\bLIMIT\b", expression, re.I).start()
                     expression = expression[:_]
 
                 elif Backend.getIdentifiedDbms() in (DBMS.MSSQL, DBMS.SYBASE):
@@ -881,12 +901,30 @@ class Agent(object):
         fromIndex = limitedQuery.index(" FROM ")
         untilFrom = limitedQuery[:fromIndex]
         fromFrom = limitedQuery[fromIndex + 1:]
-        orderBy = False
+        orderBy = None
 
         if Backend.getIdentifiedDbms() in (DBMS.MYSQL, DBMS.PGSQL, DBMS.SQLITE):
             limitStr = queries[Backend.getIdentifiedDbms()].limit.query % (num, 1)
             limitedQuery += " %s" % limitStr
 
+        elif Backend.isDbms(DBMS.HSQLDB):
+            match = re.search(r"ORDER BY [^ ]+", limitedQuery)
+            if match:
+                limitedQuery = re.sub(r"\s*%s\s*" % match.group(0), " ", limitedQuery).strip()
+                limitedQuery += " %s" % match.group(0)
+
+            if query.startswith("SELECT "):
+                limitStr = queries[Backend.getIdentifiedDbms()].limit.query % (num, 1)
+                limitedQuery = limitedQuery.replace("SELECT ", "SELECT %s " % limitStr, 1)
+            else:
+                limitStr = queries[Backend.getIdentifiedDbms()].limit.query2 % (1, num)
+                limitedQuery += " %s" % limitStr
+
+            if not match:
+                match = re.search(r"%s\s+(\w+)" % re.escape(limitStr), limitedQuery)
+                if match:
+                    orderBy = " ORDER BY %s" % match.group(1)
+
         elif Backend.isDbms(DBMS.FIREBIRD):
             limitStr = queries[Backend.getIdentifiedDbms()].limit.query % (num + 1, num + 1)
             limitedQuery += " %s" % limitStr
@@ -903,7 +941,7 @@ class Agent(object):
             else:
                 limitedQuery = "%s FROM (SELECT %s,%s" % (untilFrom, ','.join(f for f in field), limitStr)
 
-            limitedQuery = limitedQuery % fromFrom
+            limitedQuery = safeStringFormat(limitedQuery, (fromFrom,))
             limitedQuery += "=%d" % (num + 1)
 
         elif Backend.isDbms(DBMS.MSSQL):
@@ -964,12 +1002,13 @@ class Agent(object):
 
     def forgeQueryOutputLength(self, expression):
         lengthQuery = queries[Backend.getIdentifiedDbms()].length.query
-        select = re.search("\ASELECT\s+", expression, re.I)
+        select = re.search(r"\ASELECT\s+", expression, re.I)
-        selectTopExpr = re.search("\ASELECT\s+TOP\s+[\d]+\s+(.+?)\s+FROM", expression, re.I)
+        selectTopExpr = re.search(r"\ASELECT\s+TOP\s+[\d]+\s+(.+?)\s+FROM", expression, re.I)
+        selectMinMaxExpr = re.search(r"\ASELECT\s+(MIN|MAX)\(.+?\)\s+FROM", expression, re.I)
 
         _, _, _, _, _, _, fieldsStr, _ = self.getFields(expression)
 
-        if selectTopExpr:
+        if selectTopExpr or selectMinMaxExpr:
             lengthExpr = lengthQuery % ("(%s)" % expression)
         elif select:
             lengthExpr = expression.replace(fieldsStr, lengthQuery % fieldsStr, 1)
@@ -1033,7 +1072,7 @@ class Agent(object):
         """
 
         _ = re.escape(PAYLOAD_DELIMITER)
-        return re.sub("(?s)(%s.*?%s)" % (_, _), ("%s%s%s" % (PAYLOAD_DELIMITER, payload, PAYLOAD_DELIMITER)).replace("\\", r"\\"), value) if value else value
+        return re.sub("(?s)(%s.*?%s)" % (_, _), ("%s%s%s" % (PAYLOAD_DELIMITER, getUnicode(payload), PAYLOAD_DELIMITER)).replace("\\", r"\\"), value) if value else value
 
     def runAsDBMSUser(self, query):
         if conf.dbmsCred and "Ad Hoc Distributed Queries" not in query:
@@ -1041,5 +1080,20 @@ class Agent(object):
 
         return query
 
+    def whereQuery(self, query):
+        if conf.dumpWhere and query:
+            prefix, suffix = query.split(" ORDER BY ") if " ORDER BY " in query else (query, "")
+
+            if "%s)" % conf.tbl.upper() in prefix.upper():
+                prefix = re.sub(r"(?i)%s\)" % re.escape(conf.tbl), "%s WHERE %s)" % (conf.tbl, conf.dumpWhere), prefix)
+            elif re.search(r"(?i)\bWHERE\b", prefix):
+                prefix += " AND %s" % conf.dumpWhere
+            else:
+                prefix += " WHERE %s" % conf.dumpWhere
+
+            query = "%s ORDER BY %s" % (prefix, suffix) if suffix else prefix
+
+        return query
+
 # SQL agent
 agent = Agent()

lib/core/bigarray.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -15,6 +15,7 @@ import os
 import sys
 import tempfile
 
+from lib.core.enums import MKSTEMP_PREFIX
 from lib.core.exception import SqlmapSystemException
 from lib.core.settings import BIGARRAY_CHUNK_SIZE
 
@@ -79,7 +80,7 @@ class BigArray(list):
                     self.chunks[-1] = pickle.load(fp)
             except IOError, ex:
                 errMsg = "exception occurred while retrieving data "
-                errMsg += "from a temporary file ('%s')" % ex
+                errMsg += "from a temporary file ('%s')" % ex.message
                 raise SqlmapSystemException, errMsg
         return self.chunks[-1].pop()
 
@@ -91,7 +92,7 @@ class BigArray(list):
 
     def _dump(self, chunk):
         try:
-            handle, filename = tempfile.mkstemp()
+            handle, filename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.BIG_ARRAY)
             self.filenames.add(filename)
             os.close(handle)
             with open(filename, "w+b") as fp:
@@ -99,7 +100,7 @@ class BigArray(list):
             return filename
         except (OSError, IOError), ex:
             errMsg = "exception occurred while storing data "
-            errMsg += "to a temporary file ('%s'). Please " % ex
+            errMsg += "to a temporary file ('%s'). Please " % ex.message
             errMsg += "make sure that there is enough disk space left. If problem persists, "
             errMsg += "try to set environment variable 'TEMP' to a location "
             errMsg += "writeable by the current user"
@@ -115,7 +116,7 @@ class BigArray(list):
                     self.cache = Cache(index, pickle.load(fp), False)
             except IOError, ex:
                 errMsg = "exception occurred while retrieving data "
-                errMsg += "from a temporary file ('%s')" % ex
+                errMsg += "from a temporary file ('%s')" % ex.message
                 raise SqlmapSystemException, errMsg
 
     def __getstate__(self):

lib/core/convert.py

@@ -1,17 +1,26 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
+try:
+    import cPickle as pickle
+except:
+    import pickle
+finally:
+    import pickle as picklePy
+
 import base64
 import json
-import pickle
+import re
+import StringIO
 import sys
 
 from lib.core.settings import IS_WIN
 from lib.core.settings import UNICODE_ENCODING
+from lib.core.settings import PICKLE_REDUCE_WHITELIST
 
 def base64decode(value):
     """
@@ -38,7 +47,7 @@ def base64pickle(value):
     Serializes (with pickle) and encodes to Base64 format supplied (binary) value
 
     >>> base64pickle('foobar')
-    'gAJVBmZvb2JhcnEALg=='
+    'gAJVBmZvb2JhcnEBLg=='
     """
 
     retVal = None
@@ -57,20 +66,36 @@ def base64pickle(value):
 
     return retVal
 
-def base64unpickle(value):
+def base64unpickle(value, unsafe=False):
     """
     Decodes value from Base64 to plain format and deserializes (with pickle) its content
 
-    >>> base64unpickle('gAJVBmZvb2JhcnEALg==')
+    >>> base64unpickle('gAJVBmZvb2JhcnEBLg==')
     'foobar'
     """
 
     retVal = None
 
+    def _(self):
+        if len(self.stack) > 1:
+            func = self.stack[-2]
+            if func not in PICKLE_REDUCE_WHITELIST:
+                raise Exception, "abusing reduce() is bad, Mkay!"
+        self.load_reduce()
+
+    def loads(str):
+        f = StringIO.StringIO(str)
+        if unsafe:
+            unpickler = picklePy.Unpickler(f)
+            unpickler.dispatch[picklePy.REDUCE] = _
+        else:
+            unpickler = pickle.Unpickler(f)
+        return unpickler.load()
+
     try:
-        retVal = pickle.loads(base64decode(value))
+        retVal = loads(base64decode(value))
     except TypeError: 
-        retVal = pickle.loads(base64decode(bytes(value)))
+        retVal = loads(base64decode(bytes(value)))
 
     return retVal
 
@@ -143,6 +168,10 @@ def htmlunescape(value):
     if value and isinstance(value, basestring):
         codes = (('&lt;', '<'), ('&gt;', '>'), ('&quot;', '"'), ('&nbsp;', ' '), ('&amp;', '&'))
         retVal = reduce(lambda x, y: x.replace(y[0], y[1]), codes, retVal)
+        try:
+            retVal = re.sub(r"&#x([^ ;]+);", lambda match: unichr(int(match.group(1), 16)), retVal)
+        except ValueError:
+            pass
     return retVal
 
 def singleTimeWarnMessage(message):  # Cross-linked function

lib/core/data.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/core/datatype.py

@@ -1,15 +1,13 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
 import copy
 import types
 
-from lib.core.exception import SqlmapDataException
-
 class AttribDict(dict):
     """
     This class defines the sqlmap object, inheriting from Python data
@@ -43,7 +41,7 @@ class AttribDict(dict):
         try:
             return self.__getitem__(item)
         except KeyError:
-            raise SqlmapDataException("unable to access item '%s'" % item)
+            raise AttributeError("unable to access item '%s'" % item)
 
     def __setattr__(self, item, value):
         """
@@ -93,6 +91,7 @@ class InjectionDict(AttribDict):
         self.prefix = None
         self.suffix = None
         self.clause = None
+        self.notes = []  # Note: https://github.com/sqlmapproject/sqlmap/issues/1888
 
         # data is a dict with various stype, each which is a dict with
         # all the information specific for that stype

lib/core/decorators.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -15,10 +15,13 @@ def cachedmethod(f, cache={}):
     def _(*args, **kwargs):
         try:
             key = (f, tuple(args), frozenset(kwargs.items()))
+            if key not in cache:
+                cache[key] = f(*args, **kwargs)
         except:
             key = "".join(str(_) for _ in (f, args, kwargs))
-        if key not in cache:
+            if key not in cache:
-            cache[key] = f(*args, **kwargs)
+                cache[key] = f(*args, **kwargs)
+
         return cache[key]
 
     return _

lib/core/defaults.py

@@ -1,17 +1,16 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
 from lib.core.datatype import AttribDict
 
 _defaults = {
-   "csvDel":       ",",
+   "csvDel":       ',',
    "timeSec":      5,
    "googlePage":   1,
-   "cpuThrottle":  5,
    "verbose":      1,
    "delay":        0,
    "timeout":      30,
@@ -22,7 +21,7 @@ _defaults = {
    "risk":         1,
    "dumpFormat":   "CSV",
    "tech":         "BEUSTQ",
-   "torType":      "HTTP",
+   "torType":      "SOCKS5",
 }
 
 defaults = AttribDict(_defaults)

lib/core/dicts.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -21,218 +21,268 @@ from lib.core.settings import MAXDB_ALIASES
 from lib.core.settings import SYBASE_ALIASES
 from lib.core.settings import DB2_ALIASES
 from lib.core.settings import HSQLDB_ALIASES
+from lib.core.settings import INFORMIX_ALIASES
 
 FIREBIRD_TYPES = {
-                    "261": "BLOB",
+    261: "BLOB",
-                    "14": "CHAR",
+    14: "CHAR",
-                    "40": "CSTRING",
+    40: "CSTRING",
-                    "11": "D_FLOAT",
+    11: "D_FLOAT",
-                    "27": "DOUBLE",
+    27: "DOUBLE",
-                    "10": "FLOAT",
+    10: "FLOAT",
-                    "16": "INT64",
+    16: "INT64",
-                    "8": "INTEGER",
+    8: "INTEGER",
-                    "9": "QUAD",
+    9: "QUAD",
-                    "7": "SMALLINT",
+    7: "SMALLINT",
-                    "12": "DATE",
+    12: "DATE",
-                    "13": "TIME",
+    13: "TIME",
-                    "35": "TIMESTAMP",
+    35: "TIMESTAMP",
-                    "37": "VARCHAR",
+    37: "VARCHAR",
-                }
+}
+
+INFORMIX_TYPES = {
+    0: "CHAR",
+    1: "SMALLINT",
+    2: "INTEGER",
+    3: "FLOAT",
+    4: "SMALLFLOAT",
+    5: "DECIMAL",
+    6: "SERIAL",
+    7: "DATE",
+    8: "MONEY",
+    9: "NULL",
+    10: "DATETIME",
+    11: "BYTE",
+    12: "TEXT",
+    13: "VARCHAR",
+    14: "INTERVAL",
+    15: "NCHAR",
+    16: "NVARCHAR",
+    17: "INT8",
+    18: "SERIAL8",
+    19: "SET",
+    20: "MULTISET",
+    21: "LIST",
+    22: "ROW (unnamed)",
+    23: "COLLECTION",
+    40: "Variable-length opaque type",
+    41: "Fixed-length opaque type",
+    43: "LVARCHAR",
+    45: "BOOLEAN",
+    52: "BIGINT",
+    53: "BIGSERIAL",
+    2061: "IDSSECURITYLABEL",
+    4118: "ROW (named)",
+}
 
 SYBASE_TYPES = {
-                    "14": "floatn",
+    14: "floatn",
-                    "8": "float",
+    8: "float",
-                    "15": "datetimn",
+    15: "datetimn",
-                    "12": "datetime",
+    12: "datetime",
-                    "23": "real",
+    23: "real",
-                    "28": "numericn",
+    28: "numericn",
-                    "10": "numeric",
+    10: "numeric",
-                    "27": "decimaln",
+    27: "decimaln",
-                    "26": "decimal",
+    26: "decimal",
-                    "17": "moneyn",
+    17: "moneyn",
-                    "11": "money",
+    11: "money",
-                    "21": "smallmoney",
+    21: "smallmoney",
-                    "22": "smalldatetime",
+    22: "smalldatetime",
-                    "13": "intn",
+    13: "intn",
-                    "7": "int",
+    7: "int",
-                    "6": "smallint",
+    6: "smallint",
-                    "5": "tinyint",
+    5: "tinyint",
-                    "16": "bit",
+    16: "bit",
-                    "2": "varchar",
+    2: "varchar",
-                    "18": "sysname",
+    18: "sysname",
-                    "25": "nvarchar",
+    25: "nvarchar",
-                    "1": "char",
+    1: "char",
-                    "24": "nchar",
+    24: "nchar",
-                    "4": "varbinary",
+    4: "varbinary",
-                    "80": "timestamp",
+    80: "timestamp",
-                    "3": "binary",
+    3: "binary",
-                    "19": "text",
+    19: "text",
-                    "20": "image",
+    20: "image",
-                }
+}
 
 MYSQL_PRIVS = {
-                    1: "select_priv",
+    1: "select_priv",
-                    2: "insert_priv",
+    2: "insert_priv",
-                    3: "update_priv",
+    3: "update_priv",
-                    4: "delete_priv",
+    4: "delete_priv",
-                    5: "create_priv",
+    5: "create_priv",
-                    6: "drop_priv",
+    6: "drop_priv",
-                    7: "reload_priv",
+    7: "reload_priv",
-                    8: "shutdown_priv",
+    8: "shutdown_priv",
-                    9: "process_priv",
+    9: "process_priv",
-                    10: "file_priv",
+    10: "file_priv",
-                    11: "grant_priv",
+    11: "grant_priv",
-                    12: "references_priv",
+    12: "references_priv",
-                    13: "index_priv",
+    13: "index_priv",
-                    14: "alter_priv",
+    14: "alter_priv",
-                    15: "show_db_priv",
+    15: "show_db_priv",
-                    16: "super_priv",
+    16: "super_priv",
-                    17: "create_tmp_table_priv",
+    17: "create_tmp_table_priv",
-                    18: "lock_tables_priv",
+    18: "lock_tables_priv",
-                    19: "execute_priv",
+    19: "execute_priv",
-                    20: "repl_slave_priv",
+    20: "repl_slave_priv",
-                    21: "repl_client_priv",
+    21: "repl_client_priv",
-                    22: "create_view_priv",
+    22: "create_view_priv",
-                    23: "show_view_priv",
+    23: "show_view_priv",
-                    24: "create_routine_priv",
+    24: "create_routine_priv",
-                    25: "alter_routine_priv",
+    25: "alter_routine_priv",
-                    26: "create_user_priv",
+    26: "create_user_priv",
-                }
+}
 
 PGSQL_PRIVS = {
-                    1: "createdb",
+    1: "createdb",
-                    2: "super",
+    2: "super",
-                    3: "catupd",
+    3: "catupd",
-                }
+}
 
 # Reference(s): http://stackoverflow.com/a/17672504
 #               http://docwiki.embarcadero.com/InterBase/XE7/en/RDB$USER_PRIVILEGES
 
 FIREBIRD_PRIVS = {
-                    "S": "SELECT",
+    "S": "SELECT",
-                    "I": "INSERT",
+    "I": "INSERT",
-                    "U": "UPDATE",
+    "U": "UPDATE",
-                    "D": "DELETE",
+    "D": "DELETE",
-                    "R": "REFERENCE",
+    "R": "REFERENCE",
-                    "E": "EXECUTE",
+    "X": "EXECUTE",
-                    "X": "EXECUTE",
+    "A": "ALL",
-                    "A": "ALL",
+    "M": "MEMBER",
-                    "M": "MEMBER",
+    "T": "DECRYPT",
-                    "T": "DECRYPT",
+    "E": "ENCRYPT",
-                    "E": "ENCRYPT",
+    "B": "SUBSCRIBE",
-                    "B": "SUBSCRIBE",
+}
-                }
+
+# Reference(s): https://www.ibm.com/support/knowledgecenter/SSGU8G_12.1.0/com.ibm.sqls.doc/ids_sqs_0147.htm
+#               https://www.ibm.com/support/knowledgecenter/SSGU8G_11.70.0/com.ibm.sqlr.doc/ids_sqr_077.htm
+
+INFORMIX_PRIVS = {
+    "D": "DBA (all privileges)",
+    "R": "RESOURCE (create UDRs, UDTs, permanent tables and indexes)",
+    "C": "CONNECT (work with existing tables)",
+    "G": "ROLE",
+    "U": "DEFAULT (implicit connection)",
+}
 
 DB2_PRIVS = {
-                    1: "CONTROLAUTH",
+    1: "CONTROLAUTH",
-                    2: "ALTERAUTH",
+    2: "ALTERAUTH",
-                    3: "DELETEAUTH",
+    3: "DELETEAUTH",
-                    4: "INDEXAUTH",
+    4: "INDEXAUTH",
-                    5: "INSERTAUTH",
+    5: "INSERTAUTH",
-                    6: "REFAUTH",
+    6: "REFAUTH",
-                    7: "SELECTAUTH",
+    7: "SELECTAUTH",
-                    8: "UPDATEAUTH",
+    8: "UPDATEAUTH",
-           }
+}
 
 DUMP_REPLACEMENTS = {" ": NULL, "": BLANK}
 
 DBMS_DICT = {
-                DBMS.MSSQL: (MSSQL_ALIASES, "python-pymssql", "http://pymssql.sourceforge.net/", "mssql+pymssql"),
+    DBMS.MSSQL: (MSSQL_ALIASES, "python-pymssql", "https://github.com/pymssql/pymssql", "mssql+pymssql"),
-                DBMS.MYSQL: (MYSQL_ALIASES, "python pymysql", "https://github.com/petehunt/PyMySQL/", "mysql"),
+    DBMS.MYSQL: (MYSQL_ALIASES, "python-pymysql", "https://github.com/petehunt/PyMySQL/", "mysql"),
-                DBMS.PGSQL: (PGSQL_ALIASES, "python-psycopg2", "http://initd.org/psycopg/", "postgresql"),
+    DBMS.PGSQL: (PGSQL_ALIASES, "python-psycopg2", "http://initd.org/psycopg/", "postgresql"),
-                DBMS.ORACLE: (ORACLE_ALIASES, "python cx_Oracle", "http://cx-oracle.sourceforge.net/", "oracle"),
+    DBMS.ORACLE: (ORACLE_ALIASES, "python cx_Oracle", "http://cx-oracle.sourceforge.net/", "oracle"),
-                DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "http://packages.ubuntu.com/quantal/python-sqlite", "sqlite"),
+    DBMS.SQLITE: (SQLITE_ALIASES, "python-sqlite", "http://packages.ubuntu.com/quantal/python-sqlite", "sqlite"),
-                DBMS.ACCESS: (ACCESS_ALIASES, "python-pyodbc", "http://pyodbc.googlecode.com/", "access"),
+    DBMS.ACCESS: (ACCESS_ALIASES, "python-pyodbc", "https://github.com/mkleehammer/pyodbc", "access"),
-                DBMS.FIREBIRD: (FIREBIRD_ALIASES, "python-kinterbasdb", "http://kinterbasdb.sourceforge.net/", "firebird"),
+    DBMS.FIREBIRD: (FIREBIRD_ALIASES, "python-kinterbasdb", "http://kinterbasdb.sourceforge.net/", "firebird"),
-                DBMS.MAXDB: (MAXDB_ALIASES, None, None, "maxdb"),
+    DBMS.MAXDB: (MAXDB_ALIASES, None, None, "maxdb"),
-                DBMS.SYBASE: (SYBASE_ALIASES, "python-pymssql", "http://pymssql.sourceforge.net/", "sybase"),
+    DBMS.SYBASE: (SYBASE_ALIASES, "python-pymssql", "https://github.com/pymssql/pymssql", "sybase"),
-                DBMS.DB2: (DB2_ALIASES, "python ibm-db", "http://code.google.com/p/ibm-db/", "ibm_db_sa"),
+    DBMS.DB2: (DB2_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"),
-                DBMS.HSQLDB: (HSQLDB_ALIASES, "python jaydebeapi & python-jpype", "https://pypi.python.org/pypi/JayDeBeApi/ & http://jpype.sourceforge.net/", None),
+    DBMS.HSQLDB: (HSQLDB_ALIASES, "python jaydebeapi & python-jpype", "https://pypi.python.org/pypi/JayDeBeApi/ & http://jpype.sourceforge.net/", None),
-            }
+    DBMS.INFORMIX: (INFORMIX_ALIASES, "python ibm-db", "https://github.com/ibmdb/python-ibmdb", "ibm_db_sa"),
+}
 
 FROM_DUMMY_TABLE = {
-                        DBMS.ORACLE: " FROM DUAL",
+    DBMS.ORACLE: " FROM DUAL",
-                        DBMS.ACCESS: " FROM MSysAccessObjects",
+    DBMS.ACCESS: " FROM MSysAccessObjects",
-                        DBMS.FIREBIRD: " FROM RDB$DATABASE",
+    DBMS.FIREBIRD: " FROM RDB$DATABASE",
-                        DBMS.MAXDB: " FROM VERSIONS",
+    DBMS.MAXDB: " FROM VERSIONS",
-                        DBMS.DB2: " FROM SYSIBM.SYSDUMMY1",
+    DBMS.DB2: " FROM SYSIBM.SYSDUMMY1",
-                        DBMS.HSQLDB: " FROM INFORMATION_SCHEMA.SYSTEM_USERS"
+    DBMS.HSQLDB: " FROM INFORMATION_SCHEMA.SYSTEM_USERS",
-                   }
+    DBMS.INFORMIX: " FROM SYSMASTER:SYSDUAL"
+}
 
 SQL_STATEMENTS = {
-                        "SQL SELECT statement":  (
+    "SQL SELECT statement":  (
-                             "select ",
+            "select ",
-                             "show ",
+            "show ",
-                             " top ",
+            " top ",
-                             " distinct ",
+            " distinct ",
-                             " from ",
+            " from ",
-                             " from dual",
+            " from dual",
-                             " where ",
+            " where ",
-                             " group by ",
+            " group by ",
-                             " order by ",
+            " order by ",
-                             " having ",
+            " having ",
-                             " limit ",
+            " limit ",
-                             " offset ",
+            " offset ",
-                             " union all ",
+            " union all ",
-                             " rownum as ",
+            " rownum as ",
-                             "(case ",           ),
+            "(case ",        ),
 
-                         "SQL data definition":  (
+    "SQL data definition":   (
-                             "create ",
+        "create ",
-                             "declare ",
+        "declare ",
-                             "drop ",
+        "drop ",
-                             "truncate ",
+        "truncate ",
-                             "alter ",           ),
+        "alter ",            ),
 
-                        "SQL data manipulation": (
+    "SQL data manipulation": (
-                             "bulk ",
+            "bulk ",
-                             "insert ",
+            "insert ",
-                             "update ",
+            "update ",
-                             "delete ",
+            "delete ",
-                             "merge ",
+            "merge ",
-                             "load ",            ),
+            "load ",         ),
 
-                        "SQL data control":      (
+    "SQL data control":      (
-                             "grant ",
+            "grant ",
-                             "revoke ",          ),
+            "revoke ",       ),
 
-                        "SQL data execution":    (
+    "SQL data execution":    (
-                             "exec ",
+            "exec ",
-                             "execute ",
+            "execute ",
-                             "values ", 
+            "values ", 
-                             "call ",            ),
+            "call ",         ),
 
-                        "SQL transaction":       (
+    "SQL transaction":       (
-                             "start transaction ",
+            "start transaction ",
-                             "begin work ",
+            "begin work ",
-                             "begin transaction ",
+            "begin transaction ",
-                             "commit ",
+            "commit ",
-                             "rollback ",        ),
+            "rollback ",     ),
-                     }
+}
 
 POST_HINT_CONTENT_TYPES = {
-                                POST_HINT.JSON: "application/json",
+    POST_HINT.JSON: "application/json",
-                                POST_HINT.JSON_LIKE: "application/json",
+    POST_HINT.JSON_LIKE: "application/json",
-                                POST_HINT.MULTIPART: "multipart/form-data",
+    POST_HINT.MULTIPART: "multipart/form-data",
-                                POST_HINT.SOAP: "application/soap+xml",
+    POST_HINT.SOAP: "application/soap+xml",
-                                POST_HINT.XML: "application/xml",
+    POST_HINT.XML: "application/xml",
-                                POST_HINT.ARRAY_LIKE: "application/x-www-form-urlencoded; charset=utf-8",
+    POST_HINT.ARRAY_LIKE: "application/x-www-form-urlencoded; charset=utf-8",
-                          }
+}
 
 DEPRECATED_OPTIONS = {
-                        "--replicate": "use '--dump-format=SQLITE' instead",
+    "--replicate": "use '--dump-format=SQLITE' instead",
-                        "--no-unescape": "use '--no-escape' instead",
+    "--no-unescape": "use '--no-escape' instead",
-                        "--binary": "use '--binary-fields' instead",
+    "--binary": "use '--binary-fields' instead",
-                        "--check-payload": None,
+    "--auth-private": "use '--auth-file' instead",
-                        "--check-waf": None,
+    "--check-payload": None,
-                     }
+    "--check-waf": None,
+    "--pickled-options": "use '--api -c ...' instead",
+}
 
 DUMP_DATA_PREPROCESS = {
-                            DBMS.ORACLE: {"XMLTYPE": "(%s).getStringVal()"},  # Reference: https://www.tibcommunity.com/docs/DOC-3643
+    DBMS.ORACLE: {"XMLTYPE": "(%s).getStringVal()"},  # Reference: https://www.tibcommunity.com/docs/DOC-3643
-                            DBMS.MSSQL: {"IMAGE": "CONVERT(VARBINARY(MAX),%s)"},
+    DBMS.MSSQL: {"IMAGE": "CONVERT(VARBINARY(MAX),%s)"},
-                       }
+}
 
 DEFAULT_DOC_ROOTS = {
-                        OS.WINDOWS: ("C:/xampp/htdocs/", "C:/Inetpub/wwwroot/"),
+    OS.WINDOWS: ("C:/xampp/htdocs/", "C:/wamp/www/", "C:/Inetpub/wwwroot/"),
-                        OS.LINUX: ("/var/www/", "/var/www/html", "/usr/local/apache2/htdocs", "/var/www/nginx-default")  # Reference: https://wiki.apache.org/httpd/DistrosDefaultLayout
+    OS.LINUX: ("/var/www/", "/var/www/html", "/usr/local/apache2/htdocs", "/var/www/nginx-default", "/srv/www")  # Reference: https://wiki.apache.org/httpd/DistrosDefaultLayout
-                    }
+}

lib/core/dump.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -9,12 +9,15 @@ import cgi
 import hashlib
 import os
 import re
+import shutil
 import tempfile
 import threading
 
 from lib.core.common import Backend
+from lib.core.common import checkFile
 from lib.core.common import dataToDumpFile
 from lib.core.common import dataToStdout
+from lib.core.common import getSafeExString
 from lib.core.common import getUnicode
 from lib.core.common import isListLike
 from lib.core.common import normalizeUnicode
@@ -36,6 +39,7 @@ from lib.core.exception import SqlmapGenericException
 from lib.core.exception import SqlmapValueException
 from lib.core.exception import SqlmapSystemException
 from lib.core.replication import Replication
+from lib.core.settings import DUMP_FILE_BUFFER_SIZE
 from lib.core.settings import HTML_DUMP_CSS_STYLE
 from lib.core.settings import IS_WIN
 from lib.core.settings import METADB_SUFFIX
@@ -59,7 +63,7 @@ class Dump(object):
         self._lock = threading.Lock()
 
     def _write(self, data, newline=True, console=True, content_type=None):
-        if hasattr(conf, "api"):
+        if conf.api:
             dataToStdout(data, content_type=content_type, status=CONTENT_STATUS.COMPLETE)
             return
 
@@ -74,7 +78,7 @@ class Dump(object):
         try:
             self._outputFP.write(text)
         except IOError, ex:
-            errMsg = "error occurred while writing to log file ('%s')" % ex.message
+            errMsg = "error occurred while writing to log file ('%s')" % getSafeExString(ex)
             raise SqlmapGenericException(errMsg)
 
         if kb.get("multiThreadMode"):
@@ -94,7 +98,7 @@ class Dump(object):
         try:
             self._outputFP = openFile(self._outputFile, "ab" if not conf.flushSession else "wb")
         except IOError, ex:
-            errMsg = "error occurred while opening log file ('%s')" % ex.message
+            errMsg = "error occurred while opening log file ('%s')" % getSafeExString(ex)
             raise SqlmapGenericException(errMsg)
 
     def getOutputFile(self):
@@ -106,7 +110,7 @@ class Dump(object):
     def string(self, header, data, content_type=None, sort=True):
         kb.stickyLevel = None
 
-        if hasattr(conf, "api"):
+        if conf.api:
             self._write(data, content_type=content_type)
             return
 
@@ -115,9 +119,15 @@ class Dump(object):
         elif data is not None:
             _ = getUnicode(data)
 
-            if _ and _[-1] == '\n':
+            if _.endswith("\r\n"):
+                _ = _[:-2]
+
+            elif _.endswith("\n"):
                 _ = _[:-1]
 
+            if _.strip(' '):
+                _ = _.strip(' ')
+
             if "\n" in _:
                 self._write("%s:\n---\n%s\n---" % (header, _))
             else:
@@ -134,7 +144,7 @@ class Dump(object):
             except:
                 pass
 
-        if hasattr(conf, "api"):
+        if conf.api:
             self._write(elements, content_type=content_type)
             return
 
@@ -159,7 +169,7 @@ class Dump(object):
     def currentDb(self, data):
         if Backend.isDbms(DBMS.MAXDB):
             self.string("current database (no practical usage on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB)
-        elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.PGSQL):
+        elif Backend.getIdentifiedDbms() in (DBMS.ORACLE, DBMS.PGSQL, DBMS.HSQLDB):
             self.string("current schema (equivalent to database on %s)" % Backend.getIdentifiedDbms(), data, content_type=CONTENT_TYPE.CURRENT_DB)
         else:
             self.string("current database", data, content_type=CONTENT_TYPE.CURRENT_DB)
@@ -183,7 +193,7 @@ class Dump(object):
         users = userSettings.keys()
         users.sort(key=lambda x: x.lower() if isinstance(x, basestring) else x)
 
-        if hasattr(conf, "api"):
+        if conf.api:
             self._write(userSettings, content_type=content_type)
             return
 
@@ -217,7 +227,7 @@ class Dump(object):
 
     def dbTables(self, dbTables):
         if isinstance(dbTables, dict) and len(dbTables) > 0:
-            if hasattr(conf, "api"):
+            if conf.api:
                 self._write(dbTables, content_type=CONTENT_TYPE.TABLES)
                 return
 
@@ -260,7 +270,7 @@ class Dump(object):
 
     def dbTableColumns(self, tableColumns, content_type=None):
         if isinstance(tableColumns, dict) and len(tableColumns) > 0:
-            if hasattr(conf, "api"):
+            if conf.api:
                 self._write(tableColumns, content_type=content_type)
                 return
 
@@ -334,7 +344,7 @@ class Dump(object):
 
     def dbTablesCount(self, dbTables):
         if isinstance(dbTables, dict) and len(dbTables) > 0:
-            if hasattr(conf, "api"):
+            if conf.api:
                 self._write(dbTables, content_type=CONTENT_TYPE.COUNT)
                 return
 
@@ -393,17 +403,11 @@ class Dump(object):
             db = "All"
         table = tableValues["__infos__"]["table"]
 
-        if hasattr(conf, "api"):
+        if conf.api:
             self._write(tableValues, content_type=CONTENT_TYPE.DUMP_TABLE)
             return
 
-        _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(db)))
+        dumpDbPath = os.path.join(conf.dumpPath, unsafeSQLIdentificatorNaming(db))
-        if len(_) < len(db) or IS_WIN and db.upper() in WINDOWS_RESERVED_NAMES:
-            _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(db)))
-            dumpDbPath = os.path.join(conf.dumpPath, "%s-%s" % (_, hashlib.md5(unicodeencode(db)).hexdigest()[:8]))
-            warnFile = True
-        else:
-            dumpDbPath = os.path.join(conf.dumpPath, _)
 
         if conf.dumpFormat == DUMP_FORMAT.SQLITE:
             replication = Replication(os.path.join(conf.dumpPath, "%s.sqlite3" % unsafeSQLIdentificatorNaming(db)))
@@ -411,33 +415,65 @@ class Dump(object):
             if not os.path.isdir(dumpDbPath):
                 try:
                     os.makedirs(dumpDbPath, 0755)
-                except (OSError, IOError), ex:
+                except:
-                    try:
+                    warnFile = True
-                        tempDir = tempfile.mkdtemp(prefix="sqlmapdb")
-                    except IOError, _:
-                        errMsg = "unable to write to the temporary directory ('%s'). " % _
-                        errMsg += "Please make sure that your disk is not full and "
-                        errMsg += "that you have sufficient write permissions to "
-                        errMsg += "create temporary files and/or directories"
-                        raise SqlmapSystemException(errMsg)
 
-                    warnMsg = "unable to create dump directory "
+                    _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(db)))
-                    warnMsg += "'%s' (%s). " % (dumpDbPath, ex)
+                    dumpDbPath = os.path.join(conf.dumpPath, "%s-%s" % (_, hashlib.md5(unicodeencode(db)).hexdigest()[:8]))
-                    warnMsg += "Using temporary directory '%s' instead" % tempDir
-                    logger.warn(warnMsg)
 
-                    dumpDbPath = tempDir
+                    if not os.path.isdir(dumpDbPath):
+                        try:
+                            os.makedirs(dumpDbPath, 0755)
+                        except Exception, ex:
+                            try:
+                                tempDir = tempfile.mkdtemp(prefix="sqlmapdb")
+                            except IOError, _:
+                                errMsg = "unable to write to the temporary directory ('%s'). " % _
+                                errMsg += "Please make sure that your disk is not full and "
+                                errMsg += "that you have sufficient write permissions to "
+                                errMsg += "create temporary files and/or directories"
+                                raise SqlmapSystemException(errMsg)
 
-            _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(table)))
+                            warnMsg = "unable to create dump directory "
-            if len(_) < len(table) or IS_WIN and table.upper() in WINDOWS_RESERVED_NAMES:
+                            warnMsg += "'%s' (%s). " % (dumpDbPath, getSafeExString(ex))
-                _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(table)))
+                            warnMsg += "Using temporary directory '%s' instead" % tempDir
-                dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (_, hashlib.md5(unicodeencode(table)).hexdigest()[:8], conf.dumpFormat.lower()))
+                            logger.warn(warnMsg)
-                warnFile = True
+
+                            dumpDbPath = tempDir
+
+            dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (unsafeSQLIdentificatorNaming(table), conf.dumpFormat.lower()))
+            if not checkFile(dumpFileName, False):
+                try:
+                    openFile(dumpFileName, "w+b").close()
+                except SqlmapSystemException:
+                    raise
+                except:
+                    warnFile = True
+
+                    _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(table)))
+                    if len(_) < len(table) or IS_WIN and table.upper() in WINDOWS_RESERVED_NAMES:
+                        _ = unicodeencode(re.sub(r"[^\w]", "_", unsafeSQLIdentificatorNaming(table)))
+                        dumpFileName = os.path.join(dumpDbPath, "%s-%s.%s" % (_, hashlib.md5(unicodeencode(table)).hexdigest()[:8], conf.dumpFormat.lower()))
+                    else:
+                        dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (_, conf.dumpFormat.lower()))
             else:
-                dumpFileName = os.path.join(dumpDbPath, "%s.%s" % (_, conf.dumpFormat.lower()))
+                appendToFile = any((conf.limitStart, conf.limitStop))
 
-            appendToFile = os.path.isfile(dumpFileName) and any((conf.limitStart, conf.limitStop))
+                if not appendToFile:
-            dumpFP = openFile(dumpFileName, "wb" if not appendToFile else "ab")
+                    count = 1
+                    while True:
+                        candidate = "%s.%d" % (dumpFileName, count)
+                        if not checkFile(candidate, False):
+                            try:
+                                shutil.copyfile(dumpFileName, candidate)
+                            except IOError:
+                                pass
+                            finally:
+                                break
+                        else:
+                            count += 1
+
+            dumpFP = openFile(dumpFileName, "wb" if not appendToFile else "ab", buffering=DUMP_FILE_BUFFER_SIZE)
 
         count = int(tableValues["__infos__"]["count"])
         separator = str()
@@ -577,7 +613,8 @@ class Dump(object):
                                 if not os.path.isdir(dumpDbPath):
                                     os.makedirs(dumpDbPath, 0755)
 
-                                filepath = os.path.join(dumpDbPath, "%s-%d.bin" % (unsafeSQLIdentificatorNaming(column), randomInt(8)))
+                                _ = re.sub(r"[^\w]", "_", normalizeUnicode(unsafeSQLIdentificatorNaming(column)))
+                                filepath = os.path.join(dumpDbPath, "%s-%d.bin" % (_, randomInt(8)))
                                 warnMsg = "writing binary ('%s') content to file '%s' " % (mimetype, filepath)
                                 logger.warn(warnMsg)
 
@@ -629,17 +666,17 @@ class Dump(object):
                 logger.warn(msg)
 
     def dbColumns(self, dbColumnsDict, colConsider, dbs):
-        if hasattr(conf, "api"):
+        if conf.api:
             self._write(dbColumnsDict, content_type=CONTENT_TYPE.COLUMNS)
             return
 
         for column in dbColumnsDict.keys():
             if colConsider == "1":
-                colConsiderStr = "s like '%s' were" % unsafeSQLIdentificatorNaming(column)
+                colConsiderStr = "s LIKE '%s' were" % unsafeSQLIdentificatorNaming(column)
             else:
                 colConsiderStr = " '%s' was" % unsafeSQLIdentificatorNaming(column)
 
-            msg = "Column%s found in the " % colConsiderStr
+            msg = "column%s found in the " % colConsiderStr
             msg += "following databases:"
             self._write(msg)
 

lib/core/enums.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -34,6 +34,7 @@ class DBMS:
     SQLITE = "SQLite"
     SYBASE = "Sybase"
     HSQLDB = "HSQLDB"
+    INFORMIX = "Informix"
 
 class DBMS_DIRECTORY_NAME:
     ACCESS = "access"
@@ -47,6 +48,7 @@ class DBMS_DIRECTORY_NAME:
     SQLITE = "sqlite"
     SYBASE = "sybase"
     HSQLDB = "hsqldb"
+    INFORMIX = "informix"
 
 class CUSTOM_LOGGING:
     PAYLOAD = 9
@@ -81,7 +83,7 @@ class HTTPMETHOD:
     POST = "POST"
     HEAD = "HEAD"
     PUT = "PUT"
-    DELETE = "DETELE"
+    DELETE = "DELETE"
     TRACE = "TRACE"
     OPTIONS = "OPTIONS"
     CONNECT = "CONNECT"
@@ -164,19 +166,24 @@ class HTTP_HEADER:
     CONTENT_RANGE = "Content-Range"
     CONTENT_TYPE = "Content-Type"
     COOKIE = "Cookie"
-    SET_COOKIE = "Set-Cookie"
+    EXPIRES = "Expires"
     HOST = "Host"
+    IF_MODIFIED_SINCE = "If-Modified-Since"
+    LAST_MODIFIED = "Last-Modified"
     LOCATION = "Location"
     PRAGMA = "Pragma"
     PROXY_AUTHORIZATION = "Proxy-Authorization"
     PROXY_CONNECTION = "Proxy-Connection"
     RANGE = "Range"
     REFERER = "Referer"
+    REFRESH = "Refresh"  # Reference: http://stackoverflow.com/a/283794
     SERVER = "Server"
-    USER_AGENT = "User-Agent"
+    SET_COOKIE = "Set-Cookie"
     TRANSFER_ENCODING = "Transfer-Encoding"
     URI = "URI"
+    USER_AGENT = "User-Agent"
     VIA = "Via"
+    X_POWERED_BY = "X-Powered-By"
 
 class EXPECTED:
     BOOL = "bool"
@@ -190,6 +197,8 @@ class OPTION_TYPE:
 
 class HASHDB_KEYS:
     DBMS = "DBMS"
+    DBMS_FORK = "DBMS_FORK"
+    CHECK_WAF_RESULT = "CHECK_WAF_RESULT"
     CONF_TMP_PATH = "CONF_TMP_PATH"
     KB_ABS_FILE_PATHS = "KB_ABS_FILE_PATHS"
     KB_BRUTE_COLUMNS = "KB_BRUTE_COLUMNS"
@@ -197,6 +206,7 @@ class HASHDB_KEYS:
     KB_CHARS = "KB_CHARS"
     KB_DYNAMIC_MARKINGS = "KB_DYNAMIC_MARKINGS"
     KB_INJECTIONS = "KB_INJECTIONS"
+    KB_ERROR_CHUNK_LENGTH = "KB_ERROR_CHUNK_LENGTH"
     KB_XP_CMDSHELL_AVAILABLE = "KB_XP_CMDSHELL_AVAILABLE"
     OS = "OS"
 
@@ -277,31 +287,32 @@ class WEB_API:
     JSP = "jsp"
 
 class CONTENT_TYPE:
-    TECHNIQUES = 0
+    TARGET = 0
-    DBMS_FINGERPRINT = 1
+    TECHNIQUES = 1
-    BANNER = 2
+    DBMS_FINGERPRINT = 2
-    CURRENT_USER = 3
+    BANNER = 3
-    CURRENT_DB = 4
+    CURRENT_USER = 4
-    HOSTNAME = 5
+    CURRENT_DB = 5
-    IS_DBA = 6
+    HOSTNAME = 6
-    USERS = 7
+    IS_DBA = 7
-    PASSWORDS = 8
+    USERS = 8
-    PRIVILEGES = 9
+    PASSWORDS = 9
-    ROLES = 10
+    PRIVILEGES = 10
-    DBS = 11
+    ROLES = 11
-    TABLES = 12
+    DBS = 12
-    COLUMNS = 13
+    TABLES = 13
-    SCHEMA = 14
+    COLUMNS = 14
-    COUNT = 15
+    SCHEMA = 15
-    DUMP_TABLE = 16
+    COUNT = 16
-    SEARCH = 17
+    DUMP_TABLE = 17
-    SQL_QUERY = 18
+    SEARCH = 18
-    COMMON_TABLES = 19
+    SQL_QUERY = 19
-    COMMON_COLUMNS = 20
+    COMMON_TABLES = 20
-    FILE_READ = 21
+    COMMON_COLUMNS = 21
-    FILE_WRITE = 22
+    FILE_READ = 22
-    OS_CMD = 23
+    FILE_WRITE = 23
-    REG_READ = 24
+    OS_CMD = 24
+    REG_READ = 25
 
 PART_RUN_CONTENT_TYPES = {
     "checkDbms": CONTENT_TYPE.TECHNIQUES,
@@ -345,3 +356,21 @@ class AUTOCOMPLETE_TYPE:
     SQL = 0
     OS = 1
     SQLMAP = 2
+
+class NOTE:
+    FALSE_POSITIVE_OR_UNEXPLOITABLE = "false positive or unexploitable"
+
+class MKSTEMP_PREFIX:
+    HASHES = "sqlmaphashes-"
+    CRAWLER = "sqlmapcrawler-"
+    IPC = "sqlmapipc-"
+    CONFIG = "sqlmapconfig-"
+    TESTING = "sqlmaptesting-"
+    RESULTS = "sqlmapresults-"
+    COOKIE_JAR = "sqlmapcookiejar-"
+    BIG_ARRAY = "sqlmapbigarray-"
+
+class TIMEOUT_STATE:
+    NORMAL = 0
+    EXCEPTION = 1
+    TIMEOUT = 2

lib/core/exception.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/core/log.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/core/optiondict.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -37,11 +37,14 @@ optDict = {
                                "headers":           "string",
                                "authType":          "string",
                                "authCred":          "string",
-                               "authPrivate":       "string",
+                               "authFile":          "string",
+                               "ignore401":         "boolean",
+                               "ignoreProxy":       "boolean",
+                               "ignoreRedirects":   "boolean",
+                               "ignoreTimeouts":    "boolean",
                                "proxy":             "string",
                                "proxyCred":         "string",
                                "proxyFile":         "string",
-                               "ignoreProxy":       "boolean",
                                "tor":               "boolean",
                                "torPort":           "integer",
                                "torType":           "string",
@@ -74,6 +77,7 @@ optDict = {
                                "testParameter":     "string",
                                "skip":              "string",
                                "skipStatic":        "boolean",
+                               "paramExclude":      "string",
                                "dbms":              "string",
                                "dbmsCred":          "string",
                                "os":                "string",
@@ -104,7 +108,7 @@ optDict = {
                                "uCols":             "string",
                                "uChar":             "string",
                                "uFrom":             "string",
-                               "dnsName":           "string",
+                               "dnsDomain":         "string",
                                "secondOrder":       "string",
                              },
 
@@ -136,6 +140,7 @@ optDict = {
                                "tbl":               "string",
                                "col":               "string",
                                "excludeCol":        "string",
+                               "pivotColumn":       "string",
                                "dumpWhere":         "string",
                                "user":              "string",
                                "excludeSysDbs":     "boolean",
@@ -189,7 +194,9 @@ optDict = {
                                #"xmlFile":           "string",
                                "trafficFile":       "string",
                                "batch":             "boolean",
+                               "binaryFields":      "string",
                                "charset":           "string",
+                               "checkInternet":     "boolean",
                                "crawlDepth":        "integer",
                                "crawlExclude":      "string",
                                "csvDel":            "string",
@@ -198,13 +205,14 @@ optDict = {
                                "flushSession":      "boolean",
                                "forms":             "boolean",
                                "freshQueries":      "boolean",
+                               "harFile":           "string",
                                "hexConvert":        "boolean",
                                "outputDir":         "string",
                                "parseErrors":       "boolean",
-                               "pivotColumn":       "string",
+                               "saveConfig":        "string",
-                               "saveCmdline":       "boolean",
                                "scope":             "string",
                                "testFilter":        "string",
+                               "testSkip":          "string",
                                "updateAll":         "boolean",
                              },
 
@@ -216,25 +224,31 @@ optDict = {
                                "dependencies":      "boolean",
                                "disableColoring":   "boolean",
                                "googlePage":        "integer",
+                               "identifyWaf":       "boolean",
                                "mobile":            "boolean",
                                "offline":           "boolean",
-                               "pageRank":          "boolean",
                                "purgeOutput":       "boolean",
+                               "skipWaf":           "boolean",
                                "smart":             "boolean",
+                               "tmpDir":            "string",
+                               "webRoot":           "string",
                                "wizard":            "boolean",
                                "verbose":           "integer",
                              },
             "Hidden": {
                                "dummy":             "boolean",
-                               "binaryFields":      "string",
+                               "disablePrecon":     "boolean",
                                "profile":           "boolean",
-                               "cpuThrottle":       "integer",
                                "forceDns":          "boolean",
-                               "identifyWaf":       "boolean",
+                               "murphyRate":        "integer",
-                               "ignore401":         "boolean",
                                "smokeTest":         "boolean",
                                "liveTest":          "boolean",
                                "stopFail":          "boolean",
                                "runCase":           "string",
+                      },
+            "API": {
+                               "api":               "boolean",
+                               "taskid":            "string",
+                               "database":          "string",
                       }
           }

lib/core/profiling.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -26,9 +26,8 @@ def profile(profileOutputFile=None, dotOutputFile=None, imageOutputFile=None):
         import gtk
         import pydot
     except ImportError, e:
-        errMsg = "profiling requires third-party libraries (%s). " % getUnicode(e, UNICODE_ENCODING)
+        errMsg = "profiling requires third-party libraries ('%s') " % getUnicode(e, UNICODE_ENCODING)
-        errMsg += "Quick steps:%s" % os.linesep
+        errMsg += "(Hint: 'sudo apt-get install python-pydot python-pyparsing python-profiler graphviz')"
-        errMsg += "1) sudo apt-get install python-pydot python-pyparsing python-profiler graphviz"
         logger.error(errMsg)
 
         return
@@ -76,6 +75,11 @@ def profile(profileOutputFile=None, dotOutputFile=None, imageOutputFile=None):
     # Create graph image (png) by using pydot (python-pydot)
     # http://code.google.com/p/pydot/
     pydotGraph = pydot.graph_from_dot_file(dotOutputFile)
+
+    # Reference: http://stackoverflow.com/questions/38176472/graph-write-pdfiris-pdf-attributeerror-list-object-has-no-attribute-writ
+    if isinstance(pydotGraph, list):
+        pydotGraph = pydotGraph[0]
+
     pydotGraph.write_png(imageOutputFile)
 
     infoMsg = "displaying interactive graph with xdot library"
@@ -87,5 +91,4 @@ def profile(profileOutputFile=None, dotOutputFile=None, imageOutputFile=None):
     win.connect('destroy', gtk.main_quit)
     win.set_filter("dot")
     win.open_file(dotOutputFile)
-    gobject.timeout_add(1000, win.update, dotOutputFile)
     gtk.main()

lib/core/readlineng.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/core/replication.py

@@ -1,16 +1,19 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
 import sqlite3
 
 from extra.safe2bin.safe2bin import safechardecode
+from lib.core.common import getSafeExString
 from lib.core.common import unsafeSQLIdentificatorNaming
+from lib.core.exception import SqlmapConnectionException
 from lib.core.exception import SqlmapGenericException
 from lib.core.exception import SqlmapValueException
+from lib.core.settings import UNICODE_ENCODING
 
 class Replication(object):
     """
@@ -19,10 +22,15 @@ class Replication(object):
     """
 
     def __init__(self, dbpath):
-        self.dbpath = dbpath
+        try:
-        self.connection = sqlite3.connect(dbpath)
+            self.dbpath = dbpath
-        self.connection.isolation_level = None
+            self.connection = sqlite3.connect(dbpath)
-        self.cursor = self.connection.cursor()
+            self.connection.isolation_level = None
+            self.cursor = self.connection.cursor()
+        except sqlite3.OperationalError, ex:
+            errMsg = "error occurred while opening a replication "
+            errMsg += "file '%s' ('%s')" % (self.filepath, getSafeExString(ex))
+            raise SqlmapConnectionException(errMsg)
 
     class DataType:
         """
@@ -49,11 +57,16 @@ class Replication(object):
             self.name = unsafeSQLIdentificatorNaming(name)
             self.columns = columns
             if create:
-                self.execute('DROP TABLE IF EXISTS "%s"' % self.name)
+                try:
-                if not typeless:
+                    self.execute('DROP TABLE IF EXISTS "%s"' % self.name)
-                    self.execute('CREATE TABLE "%s" (%s)' % (self.name, ','.join('"%s" %s' % (unsafeSQLIdentificatorNaming(colname), coltype) for colname, coltype in self.columns)))
+                    if not typeless:
-                else:
+                        self.execute('CREATE TABLE "%s" (%s)' % (self.name, ','.join('"%s" %s' % (unsafeSQLIdentificatorNaming(colname), coltype) for colname, coltype in self.columns)))
-                    self.execute('CREATE TABLE "%s" (%s)' % (self.name, ','.join('"%s"' % unsafeSQLIdentificatorNaming(colname) for colname in self.columns)))
+                    else:
+                        self.execute('CREATE TABLE "%s" (%s)' % (self.name, ','.join('"%s"' % unsafeSQLIdentificatorNaming(colname) for colname in self.columns)))
+                except Exception, ex:
+                    errMsg = "problem occurred ('%s') while initializing the sqlite database " % getSafeExString(ex, UNICODE_ENCODING)
+                    errMsg += "located at '%s'" % self.parent.dbpath
+                    raise SqlmapGenericException(errMsg)
 
         def insert(self, values):
             """
@@ -70,7 +83,7 @@ class Replication(object):
             try:
                 self.parent.cursor.execute(sql, parameters)
             except sqlite3.OperationalError, ex:
-                errMsg = "problem occurred ('%s') while accessing sqlite database " % unicode(ex)
+                errMsg = "problem occurred ('%s') while accessing sqlite database " % getSafeExString(ex, UNICODE_ENCODING)
                 errMsg += "located at '%s'. Please make sure that " % self.parent.dbpath
                 errMsg += "it's not used by some other program"
                 raise SqlmapGenericException(errMsg)

lib/core/revision.py

@@ -1,15 +1,13 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
 import os
 import re
-
+import subprocess
-from subprocess import PIPE
-from subprocess import Popen as execute
 
 def getRevisionNumber():
     """
@@ -46,7 +44,7 @@ def getRevisionNumber():
             break
 
     if not retVal:
-        process = execute("git rev-parse --verify HEAD", shell=True, stdout=PIPE, stderr=PIPE)
+        process = subprocess.Popen("git rev-parse --verify HEAD", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
         stdout, _ = process.communicate()
         match = re.search(r"(?i)[0-9a-f]{32}", stdout or "")
         retVal = match.group(0) if match else None

lib/core/session.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -26,12 +26,14 @@ def setDbms(dbms):
     hashDBWrite(HASHDB_KEYS.DBMS, dbms)
 
     _ = "(%s)" % ("|".join([alias for alias in SUPPORTED_DBMS]))
-    _ = re.search("^%s" % _, dbms, re.I)
+    _ = re.search(r"\A%s( |\Z)" % _, dbms, re.I)
 
     if _:
         dbms = _.group(1)
 
     Backend.setDbms(dbms)
+    if kb.resolutionDbms:
+        hashDBWrite(HASHDB_KEYS.DBMS, kb.resolutionDbms)
 
     logger.info("the back-end DBMS is %s" % Backend.getDbms())
 

lib/core/settings.py

@@ -1,26 +1,28 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
 import os
+import random
 import re
 import subprocess
 import string
 import sys
-import time
+import types
 
+from lib.core.datatype import AttribDict
 from lib.core.enums import DBMS
 from lib.core.enums import DBMS_DIRECTORY_NAME
 from lib.core.enums import OS
-from lib.core.revision import getRevisionNumber
 
-# sqlmap version and site
+# sqlmap version (<major>.<minor>.<month>.<monthly commit>)
-VERSION = "1.0-dev"
+VERSION = "1.1.7.3"
-REVISION = getRevisionNumber()
+TYPE = "dev" if VERSION.count('.') > 2 and VERSION.split('.')[-1] != '0' else "stable"
-VERSION_STRING = "sqlmap/%s%s" % (VERSION, "-%s" % REVISION if REVISION else "-nongit-%s" % time.strftime("%Y%m%d", time.gmtime(os.path.getctime(__file__))))
+TYPE_COLORS = {"dev": 33, "stable": 90, "pip": 34}
+VERSION_STRING = "sqlmap/%s#%s" % ('.'.join(VERSION.split('.')[:-1]) if VERSION.count('.') > 2 and VERSION.split('.')[-1] == '0' else VERSION, TYPE)
 DESCRIPTION = "automatic SQL injection and database takeover tool"
 SITE = "http://sqlmap.org"
 ISSUES_PAGE = "https://github.com/sqlmapproject/sqlmap/issues/new"
@@ -28,20 +30,25 @@ GIT_REPOSITORY = "git://github.com/sqlmapproject/sqlmap.git"
 GIT_PAGE = "https://github.com/sqlmapproject/sqlmap"
 
 # colorful banner
-BANNER = """\033[01;33m         _
+BANNER = """\033[01;33m\
- ___ ___| |_____ ___ ___  \033[01;37m{\033[01;%dm%s\033[01;37m}\033[01;33m
+        ___
-|_ -| . | |     | .'| . |
+       __H__
-|___|_  |_|_|_|_|__,|  _|
+ ___ ___[.]_____ ___ ___  \033[01;37m{\033[01;%dm%s\033[01;37m}\033[01;33m
-      |_|           |_|   \033[0m\033[4;37m%s\033[0m\n
+|_ -| . [.]     | .'| . |
-""" % ((31 + hash(REVISION) % 6) if REVISION else 30, VERSION_STRING.split('/')[-1], SITE)
+|___|_  [.]_|_|_|__,|  _|
+      |_|V          |_|   \033[0m\033[4;37m%s\033[0m\n
+""" % (TYPE_COLORS.get(TYPE, 31), VERSION_STRING.split('/')[-1], SITE)
 
 # Minimum distance of ratio from kb.matchRatio to result in True
 DIFF_TOLERANCE = 0.05
 CONSTANT_RATIO = 0.9
 
-# Ratio used in heuristic check for WAF/IDS/IPS protected targets
+# Ratio used in heuristic check for WAF/IPS/IDS protected targets
 IDS_WAF_CHECK_RATIO = 0.5
 
+# Timeout used in heuristic check for WAF/IPS/IDS protected targets
+IDS_WAF_CHECK_TIMEOUT = 10
+
 # Lower and upper values for match ratio in case of stable page
 LOWER_RATIO_BOUND = 0.02
 UPPER_RATIO_BOUND = 0.98
@@ -55,11 +62,19 @@ PARTIAL_HEX_VALUE_MARKER = "__PARTIAL_HEX_VALUE__"
 URI_QUESTION_MARKER = "__QUESTION_MARK__"
 ASTERISK_MARKER = "__ASTERISK_MARK__"
 REPLACEMENT_MARKER = "__REPLACEMENT_MARK__"
+BOUNDED_INJECTION_MARKER = "__BOUNDED_INJECTION_MARK__"
+
+RANDOM_INTEGER_MARKER = "[RANDINT]"
+RANDOM_STRING_MARKER = "[RANDSTR]"
+SLEEP_TIME_MARKER = "[SLEEPTIME]"
 
 PAYLOAD_DELIMITER = "__PAYLOAD_DELIMITER__"
 CHAR_INFERENCE_MARK = "%c"
 PRINTABLE_CHAR_REGEX = r"[^\x00-\x1f\x7f-\xff]"
 
+# Regular expression used for extraction of table names (useful for (e.g.) MsAccess)
+SELECT_FROM_TABLE_REGEX = r"\bSELECT .+? FROM (?P<result>([\w.]|`[^`<>]+`)+)"
+
 # Regular expression used for recognition of textual content-type
 TEXT_CONTENT_TYPE_REGEX = r"(?i)(text|form|message|xml|javascript|ecmascript|json)"
 
@@ -69,17 +84,32 @@ PERMISSION_DENIED_REGEX = r"(command|permission|access)\s*(was|is)?\s*denied"
 # Regular expression used for recognition of generic maximum connection messages
 MAX_CONNECTIONS_REGEX = r"max.+connections"
 
+# Maximum consecutive connection errors before asking the user if he wants to continue
+MAX_CONSECUTIVE_CONNECTION_ERRORS = 15
+
+# Timeout before the pre-connection candidate is being disposed (because of high probability that the web server will reset it)
+PRECONNECT_CANDIDATE_TIMEOUT = 10
+
+# Maximum sleep time in "Murphy" (testing) mode
+MAX_MURPHY_SLEEP_TIME = 3
+
 # Regular expression used for extracting results from Google search
-GOOGLE_REGEX = r"url\?\w+=((?![^>]+webcache\.googleusercontent\.com)http[^>]+)&(sa=U|rct=j)"
+GOOGLE_REGEX = r"webcache\.googleusercontent\.com/search\?q=cache:[^:]+:([^+]+)\+&amp;cd=|url\?\w+=((?![^>]+webcache\.googleusercontent\.com)http[^>]+)&(sa=U|rct=j)"
 
 # Regular expression used for extracting results from DuckDuckGo search
 DUCKDUCKGO_REGEX = r'"u":"([^"]+)'
 
+# Regular expression used for extracting results from Disconnect Search
+DISCONNECT_SEARCH_REGEX = r'<p class="url wrapword">([^<]+)</p>'
+
+# Dummy user agent for search (if default one returns different results)
+DUMMY_SEARCH_USER_AGENT = "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:49.0) Gecko/20100101 Firefox/49.0"
+
 # Regular expression used for extracting content from "textual" tags
 TEXT_TAG_REGEX = r"(?si)<(abbr|acronym|b|blockquote|br|center|cite|code|dt|em|font|h\d|i|li|p|pre|q|strong|sub|sup|td|th|title|tt|u)(?!\w).*?>(?P<result>[^<]+)"
 
 # Regular expression used for recognition of IP addresses
-IP_ADDRESS_REGEX = r"\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b"
+IP_ADDRESS_REGEX = r"\b(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\b"
 
 # Regular expression used for recognition of generic "your ip has been blocked" messages
 BLOCKED_IP_REGEX = r"(?i)(\A|\b)ip\b.*\b(banned|blocked|block list|firewall)"
@@ -107,7 +137,7 @@ UNION_STDEV_COEFF = 7
 TIME_DELAY_CANDIDATES = 3
 
 # Default value for HTTP Accept header
-HTTP_ACCEPT_HEADER_VALUE = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
+HTTP_ACCEPT_HEADER_VALUE = "*/*"
 
 # Default value for HTTP Accept-Encoding header
 HTTP_ACCEPT_ENCODING_HEADER_VALUE = "gzip,deflate"
@@ -115,6 +145,9 @@ HTTP_ACCEPT_ENCODING_HEADER_VALUE = "gzip,deflate"
 # Default timeout for running commands over backdoor
 BACKDOOR_RUN_CMD_TIMEOUT = 5
 
+# Number of seconds to wait for thread finalization at program end
+THREAD_FINALIZATION_TIMEOUT = 1
+
 # Maximum number of techniques used in inject.py/getValue() per one value
 MAX_TECHNIQUES_PER_VALUE = 2
 
@@ -124,6 +157,9 @@ MAX_BUFFERED_PARTIAL_UNION_LENGTH = 1024
 # Suffix used for naming meta databases in DBMS(es) without explicit database name
 METADB_SUFFIX = "_masterdb"
 
+# Number of times to retry the pushValue during the exceptions (e.g. KeyboardInterrupt)
+PUSH_VALUE_EXCEPTION_RETRY_COUNT = 3
+
 # Minimum time response set needed for time-comparison based on standard deviation
 MIN_TIME_RESPONSES = 30
 
@@ -172,26 +208,20 @@ PYVERSION = sys.version.split()[0]
 
 # DBMS system databases
 MSSQL_SYSTEM_DBS = ("Northwind", "master", "model", "msdb", "pubs", "tempdb")
-MYSQL_SYSTEM_DBS = ("information_schema", "mysql")                   # Before MySQL 5.0 only "mysql"
+MYSQL_SYSTEM_DBS = ("information_schema", "mysql", "performance_schema")
-PGSQL_SYSTEM_DBS = ("information_schema", "pg_catalog", "pg_toast")
+PGSQL_SYSTEM_DBS = ("information_schema", "pg_catalog", "pg_toast", "pgagent")
-ORACLE_SYSTEM_DBS = ("CTXSYS", "DBSNMP", "DMSYS", "EXFSYS", "MDSYS", "OLAPSYS", "ORDSYS", "OUTLN", "SYS", "SYSAUX", "SYSMAN", "SYSTEM", "TSMSYS", "WMSYS", "XDB")                      # These are TABLESPACE_NAME
+ORACLE_SYSTEM_DBS = ("ANONYMOUS", "APEX_PUBLIC_USER", "CTXSYS", "DBSNMP", "DIP", "EXFSYS", "FLOWS_%", "FLOWS_FILES", "LBACSYS", "MDDATA", "MDSYS", "MGMT_VIEW", "OLAPSYS", "ORACLE_OCM", "ORDDATA", "ORDPLUGINS", "ORDSYS", "OUTLN", "OWBSYS", "SI_INFORMTN_SCHEMA", "SPATIAL_CSW_ADMIN_USR", "SPATIAL_WFS_ADMIN_USR", "SYS", "SYSMAN", "SYSTEM", "WKPROXY", "WKSYS", "WK_TEST", "WMSYS", "XDB", "XS$NULL")  # Reference: https://blog.vishalgupta.com/2011/06/19/predefined-oracle-system-schemas/ 
 SQLITE_SYSTEM_DBS = ("sqlite_master", "sqlite_temp_master")
-ACCESS_SYSTEM_DBS = ("MSysAccessObjects", "MSysACEs", "MSysObjects", "MSysQueries", "MSysRelationships", "MSysAccessStorage",\
+ACCESS_SYSTEM_DBS = ("MSysAccessObjects", "MSysACEs", "MSysObjects", "MSysQueries", "MSysRelationships", "MSysAccessStorage", "MSysAccessXML", "MSysModules", "MSysModules2")
-                        "MSysAccessXML", "MSysModules", "MSysModules2")
+FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CONSTRAINTS", "RDB$COLLATIONS", "RDB$DATABASE", "RDB$DEPENDENCIES", "RDB$EXCEPTIONS", "RDB$FIELDS", "RDB$FIELD_DIMENSIONS", " RDB$FILES", "RDB$FILTERS", "RDB$FORMATS", "RDB$FUNCTIONS", "RDB$FUNCTION_ARGUMENTS", "RDB$GENERATORS", "RDB$INDEX_SEGMENTS", "RDB$INDICES", "RDB$LOG_FILES", "RDB$PAGES", "RDB$PROCEDURES", "RDB$PROCEDURE_PARAMETERS", "RDB$REF_CONSTRAINTS", "RDB$RELATIONS", "RDB$RELATION_CONSTRAINTS", "RDB$RELATION_FIELDS", "RDB$ROLES", "RDB$SECURITY_CLASSES", "RDB$TRANSACTIONS", "RDB$TRIGGERS", "RDB$TRIGGER_MESSAGES", "RDB$TYPES", "RDB$USER_PRIVILEGES", "RDB$VIEW_RELATIONS")
-FIREBIRD_SYSTEM_DBS = ("RDB$BACKUP_HISTORY", "RDB$CHARACTER_SETS", "RDB$CHECK_CONSTRAINTS", "RDB$COLLATIONS", "RDB$DATABASE",\
-                        "RDB$DEPENDENCIES", "RDB$EXCEPTIONS", "RDB$FIELDS", "RDB$FIELD_DIMENSIONS", " RDB$FILES", "RDB$FILTERS",\
-                        "RDB$FORMATS", "RDB$FUNCTIONS", "RDB$FUNCTION_ARGUMENTS", "RDB$GENERATORS", "RDB$INDEX_SEGMENTS", "RDB$INDICES",\
-                        "RDB$LOG_FILES", "RDB$PAGES", "RDB$PROCEDURES", "RDB$PROCEDURE_PARAMETERS", "RDB$REF_CONSTRAINTS", "RDB$RELATIONS",\
-                        "RDB$RELATION_CONSTRAINTS", "RDB$RELATION_FIELDS", "RDB$ROLES", "RDB$SECURITY_CLASSES", "RDB$TRANSACTIONS", "RDB$TRIGGERS",\
-                        "RDB$TRIGGER_MESSAGES", "RDB$TYPES", "RDB$USER_PRIVILEGES", "RDB$VIEW_RELATIONS")
 MAXDB_SYSTEM_DBS = ("SYSINFO", "DOMAIN")
 SYBASE_SYSTEM_DBS = ("master", "model", "sybsystemdb", "sybsystemprocs")
-DB2_SYSTEM_DBS = ("NULLID", "SQLJ", "SYSCAT", "SYSFUN", "SYSIBM", "SYSIBMADM", "SYSIBMINTERNAL", "SYSIBMTS",\
+DB2_SYSTEM_DBS = ("NULLID", "SQLJ", "SYSCAT", "SYSFUN", "SYSIBM", "SYSIBMADM", "SYSIBMINTERNAL", "SYSIBMTS", "SYSPROC", "SYSPUBLIC", "SYSSTAT", "SYSTOOLS")
-                   "SYSPROC", "SYSPUBLIC", "SYSSTAT", "SYSTOOLS")
 HSQLDB_SYSTEM_DBS = ("INFORMATION_SCHEMA", "SYSTEM_LOB")
+INFORMIX_SYSTEM_DBS = ("sysmaster", "sysutils", "sysuser", "sysadmin")
 
 MSSQL_ALIASES = ("microsoft sql server", "mssqlserver", "mssql", "ms")
-MYSQL_ALIASES = ("mysql", "my")
+MYSQL_ALIASES = ("mysql", "my", "mariadb", "maria")
 PGSQL_ALIASES = ("postgresql", "postgres", "pgsql", "psql", "pg")
 ORACLE_ALIASES = ("oracle", "orcl", "ora", "or")
 SQLITE_ALIASES = ("sqlite", "sqlite3")
@@ -201,10 +231,11 @@ MAXDB_ALIASES = ("maxdb", "sap maxdb", "sap db")
 SYBASE_ALIASES = ("sybase", "sybase sql server")
 DB2_ALIASES = ("db2", "ibm db2", "ibmdb2")
 HSQLDB_ALIASES = ("hsql", "hsqldb", "hs", "hypersql")
+INFORMIX_ALIASES = ("informix", "ibm informix", "ibminformix")
 
 DBMS_DIRECTORY_DICT = dict((getattr(DBMS, _), getattr(DBMS_DIRECTORY_NAME, _)) for _ in dir(DBMS) if not _.startswith("_"))
 
-SUPPORTED_DBMS = MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES
+SUPPORTED_DBMS = MSSQL_ALIASES + MYSQL_ALIASES + PGSQL_ALIASES + ORACLE_ALIASES + SQLITE_ALIASES + ACCESS_ALIASES + FIREBIRD_ALIASES + MAXDB_ALIASES + SYBASE_ALIASES + DB2_ALIASES + HSQLDB_ALIASES + INFORMIX_ALIASES
 SUPPORTED_OS = ("linux", "windows")
 
 DBMS_ALIASES = ((DBMS.MSSQL, MSSQL_ALIASES), (DBMS.MYSQL, MYSQL_ALIASES), (DBMS.PGSQL, PGSQL_ALIASES), (DBMS.ORACLE, ORACLE_ALIASES), (DBMS.SQLITE, SQLITE_ALIASES), (DBMS.ACCESS, ACCESS_ALIASES), (DBMS.FIREBIRD, FIREBIRD_ALIASES), (DBMS.MAXDB, MAXDB_ALIASES), (DBMS.SYBASE, SYBASE_ALIASES), (DBMS.DB2, DB2_ALIASES), (DBMS.HSQLDB, HSQLDB_ALIASES))
@@ -213,44 +244,46 @@ USER_AGENT_ALIASES = ("ua", "useragent", "user-agent")
 REFERER_ALIASES = ("ref", "referer", "referrer")
 HOST_ALIASES = ("host",)
 
+HSQLDB_DEFAULT_SCHEMA = "PUBLIC"
+
 # Names that can't be used to name files on Windows OS
 WINDOWS_RESERVED_NAMES = ("CON", "PRN", "AUX", "NUL", "COM1", "COM2", "COM3", "COM4", "COM5", "COM6", "COM7", "COM8", "COM9", "LPT1", "LPT2", "LPT3", "LPT4", "LPT5", "LPT6", "LPT7", "LPT8", "LPT9")
 
 # Items displayed in basic help (-h) output
 BASIC_HELP_ITEMS = (
-                        "url",
+    "url",
-                        "googleDork",
+    "googleDork",
-                        "data",
+    "data",
-                        "cookie",
+    "cookie",
-                        "randomAgent",
+    "randomAgent",
-                        "proxy",
+    "proxy",
-                        "testParameter",
+    "testParameter",
-                        "dbms",
+    "dbms",
-                        "level",
+    "level",
-                        "risk",
+    "risk",
-                        "tech",
+    "tech",
-                        "getAll",
+    "getAll",
-                        "getBanner",
+    "getBanner",
-                        "getCurrentUser",
+    "getCurrentUser",
-                        "getCurrentDb",
+    "getCurrentDb",
-                        "getPasswordHashes",
+    "getPasswordHashes",
-                        "getTables",
+    "getTables",
-                        "getColumns",
+    "getColumns",
-                        "getSchema",
+    "getSchema",
-                        "dumpTable",
+    "dumpTable",
-                        "dumpAll",
+    "dumpAll",
-                        "db",
+    "db",
-                        "tbl",
+    "tbl",
-                        "col",
+    "col",
-                        "osShell",
+    "osShell",
-                        "osPwn",
+    "osPwn",
-                        "batch",
+    "batch",
-                        "checkTor",
+    "checkTor",
-                        "flushSession",
+    "flushSession",
-                        "tor",
+    "tor",
-                        "sqlmapShell",
+    "sqlmapShell",
-                        "wizard",
+    "wizard",
-                   )
+)
 
 # String representation for NULL value
 NULL = "NULL"
@@ -261,13 +294,19 @@ BLANK = "<blank>"
 # String representation for current database
 CURRENT_DB = "CD"
 
+# Regular expressions used for finding file paths in error messages
+FILE_PATH_REGEXES = (r"<b>(?P<result>[^<>]+?)</b> on line \d+", r"(?P<result>[^<>'\"]+?)['\"]? on line \d+", r"(?:[>(\[\s])(?P<result>[A-Za-z]:[\\/][\w. \\/-]*)", r"(?:[>(\[\s])(?P<result>/\w[/\w.-]+)", r"href=['\"]file://(?P<result>/[^'\"]+)")
+
 # Regular expressions used for parsing error messages (--parse-errors)
 ERROR_PARSING_REGEXES = (
-                          r"<b>[^<]*(fatal|error|warning|exception)[^<]*</b>:?\s*(?P<result>.+?)<br\s*/?\s*>",
+    r"<b>[^<]*(fatal|error|warning|exception)[^<]*</b>:?\s*(?P<result>.+?)<br\s*/?\s*>",
-                          r"(?m)^(fatal|error|warning|exception):?\s*(?P<result>.+?)$",
+    r"(?m)^(fatal|error|warning|exception):?\s*(?P<result>[^\n]+?)$",
-                          r"<li>Error Type:<br>(?P<result>.+?)</li>",
+    r"(?P<result>[^\n>]*SQL Syntax[^\n<]+)",
-                          r"error '[0-9a-f]{8}'((<[^>]+>)|\s)+(?P<result>[^<>]+)",
+    r"<li>Error Type:<br>(?P<result>.+?)</li>",
-                        )
+    r"CDbCommand (?P<result>[^<>\n]*SQL[^<>\n]+)",
+    r"error '[0-9a-f]{8}'((<[^>]+>)|\s)+(?P<result>[^<>]+)",
+    r"\[[^\n\]]+(ODBC|JDBC)[^\n\]]+\](\[[^\]]+\])?(?P<result>[^\n]+(in query expression|\(SQL| at /[^ ]+pdo)[^\n<]+)"
+)
 
 # Regular expression used for parsing charset info from meta html headers
 META_CHARSET_REGEX = r'(?si)<head>.*<meta[^>]+charset="?(?P<result>[^"> ]+).*</head>'
@@ -305,6 +344,9 @@ URI_INJECTABLE_REGEX = r"//[^/]*/([^\.*?]+)\Z"
 # Regex used for masking sensitive data
 SENSITIVE_DATA_REGEX = "(\s|=)(?P<result>[^\s=]*%s[^\s]*)\s"
 
+# Options to explicitly mask in anonymous (unhandled exception) reports (along with anything carrying the <hostname> inside)
+SENSITIVE_OPTIONS = ("hostname", "data", "dnsDomain", "googleDork", "authCred", "proxyCred", "tbl", "db", "col", "user", "cookie", "proxy", "rFile", "wFile", "dFile", "testParameter", "authCred")
+
 # Maximum number of threads (avoiding connection issues and/or DoS)
 MAX_NUMBER_OF_THREADS = 10
 
@@ -317,17 +359,20 @@ MIN_RATIO = 0.0
 # Maximum value for comparison ratio
 MAX_RATIO = 1.0
 
+# Minimum length of sentence for automatic choosing of --string (in case of high matching ratio)
+CANDIDATE_SENTENCE_MIN_LENGTH = 10
+
 # Character used for marking injectable position inside provided data
 CUSTOM_INJECTION_MARK_CHAR = '*'
 
 # Other way to declare injection position
 INJECT_HERE_MARK = '%INJECT HERE%'
 
-# Maximum length used for retrieving data over MySQL error based payload due to "known" problems with longer result strings
+# Minimum chunk length used for retrieving data over error based payloads
-MYSQL_ERROR_CHUNK_LENGTH = 50
+MIN_ERROR_CHUNK_LENGTH = 8
 
-# Maximum length used for retrieving data over MSSQL error based payload due to trimming problems with longer result strings
+# Maximum chunk length used for retrieving data over error based payloads
-MSSQL_ERROR_CHUNK_LENGTH = 100
+MAX_ERROR_CHUNK_LENGTH = 1024
 
 # Do not escape the injected statement if it contains any of the following SQL keywords
 EXCLUDE_UNESCAPE = ("WAITFOR DELAY ", " INTO DUMPFILE ", " INTO OUTFILE ", "CREATE ", "BULK ", "EXEC ", "RECONFIGURE ", "DECLARE ", "'%s'" % CHAR_INFERENCE_MARK)
@@ -341,6 +386,9 @@ REFLECTED_BORDER_REGEX = r"[^A-Za-z]+"
 # Regular expression used for replacing non-alphanum characters
 REFLECTED_REPLACEMENT_REGEX = r".+"
 
+# Maximum time (in seconds) spent per reflective value(s) replacement
+REFLECTED_REPLACEMENT_TIMEOUT = 3
+
 # Maximum number of alpha-numerical parts in reflected regex (for speed purposes)
 REFLECTED_MAX_REGEX_PARTS = 10
 
@@ -360,10 +408,10 @@ HASH_MOD_ITEM_DISPLAY = 11
 MAX_INT = sys.maxint
 
 # Options that need to be restored in multiple targets run mode
-RESTORE_MERGED_OPTIONS = ("col", "db", "dnsName", "privEsc", "tbl", "regexp", "string", "textOnly", "threads", "timeSec", "tmpPath", "uChar", "user")
+RESTORE_MERGED_OPTIONS = ("col", "db", "dnsDomain", "privEsc", "tbl", "regexp", "string", "textOnly", "threads", "timeSec", "tmpPath", "uChar", "user")
 
 # Parameters to be ignored in detection phase (upper case)
-IGNORE_PARAMETERS = ("__VIEWSTATE", "__VIEWSTATEENCRYPTED", "__EVENTARGUMENT", "__EVENTTARGET", "__EVENTVALIDATION", "ASPSESSIONID", "ASP.NET_SESSIONID", "JSESSIONID", "CFID", "CFTOKEN")
+IGNORE_PARAMETERS = ("__VIEWSTATE", "__VIEWSTATEENCRYPTED", "__VIEWSTATEGENERATOR", "__EVENTARGUMENT", "__EVENTTARGET", "__EVENTVALIDATION", "ASPSESSIONID", "ASP.NET_SESSIONID", "JSESSIONID", "CFID", "CFTOKEN")
 
 # Regular expression used for recognition of ASP.NET control parameters
 ASP_NET_CONTROL_REGEX = r"(?i)\Actl\d+\$"
@@ -386,13 +434,16 @@ CODECS_LIST_PAGE = "http://docs.python.org/library/codecs.html#standard-encoding
 # Simple regular expression used to distinguish scalar from multiple-row commands (not sole condition)
 SQL_SCALAR_REGEX = r"\A(SELECT(?!\s+DISTINCT\(?))?\s*\w*\("
 
+# Option/switch values to ignore during configuration save
+IGNORE_SAVE_OPTIONS = ("saveConfig",)
+
 # IP address of the localhost
 LOCALHOST = "127.0.0.1"
 
-# Default port used by Tor
+# Default SOCKS ports used by Tor
-DEFAULT_TOR_SOCKS_PORT = 9050
+DEFAULT_TOR_SOCKS_PORTS = (9050, 9150)
 
-# Default ports used in Tor proxy bundles
+# Default HTTP ports used by Tor
 DEFAULT_TOR_HTTP_PORTS = (8123, 8118)
 
 # Percentage below which comparison engine could have problems
@@ -413,6 +464,8 @@ HTML_TITLE_REGEX = "<title>(?P<result>[^<]+)</title>"
 # Table used for Base64 conversion in WordPress hash cracking routine
 ITOA64 = "./0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
 
+PICKLE_REDUCE_WHITELIST = (types.BooleanType, types.DictType, types.FloatType, types.IntType, types.ListType, types.LongType, types.NoneType, types.StringType, types.TupleType, types.UnicodeType, types.XRangeType, type(AttribDict()), type(set()))
+
 # Chars used to quickly distinguish if the user provided tainted parameter values
 DUMMY_SQL_INJECTION_CHARS = ";()'"
 
@@ -420,7 +473,7 @@ DUMMY_SQL_INJECTION_CHARS = ";()'"
 DUMMY_USER_INJECTION = r"(?i)[^\w](AND|OR)\s+[^\s]+[=><]|\bUNION\b.+\bSELECT\b|\bSELECT\b.+\bFROM\b|\b(CONCAT|information_schema|SLEEP|DELAY)\b"
 
 # Extensions skipped by crawler
-CRAWL_EXCLUDE_EXTENSIONS = ("gif", "jpg", "jpeg", "image", "jar", "tif", "bmp", "war", "ear", "mpg", "mpeg", "wmv", "mpeg", "scm", "iso", "dmp", "dll", "cab", "so", "avi", "mkv", "bin", "iso", "tar", "png", "pdf", "ps", "wav", "mp3", "mp4", "au", "aiff", "aac", "zip", "rar", "7z", "gz", "flv", "mov", "doc", "docx", "xls", "dot", "dotx", "xlt", "xlsx", "ppt", "pps", "pptx")
+CRAWL_EXCLUDE_EXTENSIONS = ("3ds", "3g2", "3gp", "7z", "DS_Store", "a", "aac", "adp", "ai", "aif", "aiff", "apk", "ar", "asf", "au", "avi", "bak", "bin", "bk", "bmp", "btif", "bz2", "cab", "caf", "cgm", "cmx", "cpio", "cr2", "dat", "deb", "djvu", "dll", "dmg", "dmp", "dng", "doc", "docx", "dot", "dotx", "dra", "dsk", "dts", "dtshd", "dvb", "dwg", "dxf", "ear", "ecelp4800", "ecelp7470", "ecelp9600", "egg", "eol", "eot", "epub", "exe", "f4v", "fbs", "fh", "fla", "flac", "fli", "flv", "fpx", "fst", "fvt", "g3", "gif", "gz", "h261", "h263", "h264", "ico", "ief", "image", "img", "ipa", "iso", "jar", "jpeg", "jpg", "jpgv", "jpm", "jxr", "ktx", "lvp", "lz", "lzma", "lzo", "m3u", "m4a", "m4v", "mar", "mdi", "mid", "mj2", "mka", "mkv", "mmr", "mng", "mov", "movie", "mp3", "mp4", "mp4a", "mpeg", "mpg", "mpga", "mxu", "nef", "npx", "o", "oga", "ogg", "ogv", "otf", "pbm", "pcx", "pdf", "pea", "pgm", "pic", "png", "pnm", "ppm", "pps", "ppt", "pptx", "ps", "psd", "pya", "pyc", "pyo", "pyv", "qt", "rar", "ras", "raw", "rgb", "rip", "rlc", "rz", "s3m", "s7z", "scm", "scpt", "sgi", "shar", "sil", "smv", "so", "sub", "swf", "tar", "tbz2", "tga", "tgz", "tif", "tiff", "tlz", "ts", "ttf", "uvh", "uvi", "uvm", "uvp", "uvs", "uvu", "viv", "vob", "war", "wav", "wax", "wbmp", "wdp", "weba", "webm", "webp", "whl", "wm", "wma", "wmv", "wmx", "woff", "woff2", "wvx", "xbm", "xif", "xls", "xlsx", "xlt", "xm", "xpi", "xpm", "xwd", "xz", "z", "zip", "zipx")
 
 # Patterns often seen in HTTP headers containing custom injection marking character
 PROBLEMATIC_CUSTOM_INJECTION_PATTERNS = r"(;q=[^;']+)|(\*/\*)"
@@ -431,17 +484,26 @@ BRUTE_TABLE_EXISTS_TEMPLATE = "EXISTS(SELECT %d FROM %s)"
 # Template used for common column existence check
 BRUTE_COLUMN_EXISTS_TEMPLATE = "EXISTS(SELECT %s FROM %s)"
 
-# Payload used for checking of existence of IDS/WAF (dummier the better)
+# Payload used for checking of existence of IDS/IPS/WAF (dummier the better)
-IDS_WAF_CHECK_PAYLOAD = "AND 1=1 UNION ALL SELECT 1,2,3,table_name FROM information_schema.tables WHERE 2>1-- ../../../etc/passwd"
+IDS_WAF_CHECK_PAYLOAD = "AND 1=1 UNION ALL SELECT 1,NULL,'<script>alert(\"XSS\")</script>',table_name FROM information_schema.tables WHERE 2>1--/**/; EXEC xp_cmdshell('cat ../../../etc/passwd')#"
 
-# Vectors used for provoking specific WAF/IDS/IPS behavior(s)
+# Data inside shellcodeexec to be filled with random string
+SHELLCODEEXEC_RANDOM_STRING_MARKER = "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
+
+# Generic address for checking the Internet connection while using switch --check-internet
+CHECK_INTERNET_ADDRESS = "http://ipinfo.io/"
+
+# Value to look for in response to CHECK_INTERNET_ADDRESS
+CHECK_INTERNET_VALUE = "IP Address Details"
+
+# Vectors used for provoking specific WAF/IPS/IDS behavior(s)
 WAF_ATTACK_VECTORS = (
-                        "",  # NIL
+    "",  # NIL
-                        "search=<script>alert(1)</script>",
+    "search=<script>alert(1)</script>",
-                        "file=../../../../etc/passwd",
+    "file=../../../../etc/passwd",
-                        "q=<invalid>foobar",
+    "q=<invalid>foobar",
-                        "id=1 %s" % IDS_WAF_CHECK_PAYLOAD
+    "id=1 %s" % IDS_WAF_CHECK_PAYLOAD
-                     )
+)
 
 # Used for status representation in dictionary attack phase
 ROTATING_CHARS = ('\\', '|', '|', '/', '-')
@@ -449,29 +511,36 @@ ROTATING_CHARS = ('\\', '|', '|', '/', '-')
 # Approximate chunk length (in bytes) used by BigArray objects (only last chunk and cached one are held in memory)
 BIGARRAY_CHUNK_SIZE = 1024 * 1024
 
+# Maximum number of socket pre-connects
+SOCKET_PRE_CONNECT_QUEUE_SIZE = 3
+
 # Only console display last n table rows
 TRIM_STDOUT_DUMP_SIZE = 256
 
+# Reference: http://stackoverflow.com/a/3168436
+# Reference: https://support.microsoft.com/en-us/kb/899149
+DUMP_FILE_BUFFER_SIZE = 1024
+
 # Parse response headers only first couple of times
 PARSE_HEADERS_LIMIT = 3
 
 # Step used in ORDER BY technique used for finding the right number of columns in UNION query injections
 ORDER_BY_STEP = 10
 
-# Maximum number of times for revalidation of a character in time-based injections
+# Maximum number of times for revalidation of a character in inference (as required)
-MAX_TIME_REVALIDATION_STEPS = 5
+MAX_REVALIDATION_STEPS = 5
 
 # Characters that can be used to split parameter values in provided command line (e.g. in --tamper)
-PARAMETER_SPLITTING_REGEX = r'[,|;]'
+PARAMETER_SPLITTING_REGEX = r"[,|;]"
 
 # Regular expression describing possible union char value (e.g. used in --union-char)
-UNION_CHAR_REGEX = r'\A\w+\Z'
+UNION_CHAR_REGEX = r"\A\w+\Z"
 
 # Attribute used for storing original parameter value in special cases (e.g. POST)
-UNENCODED_ORIGINAL_VALUE = 'original'
+UNENCODED_ORIGINAL_VALUE = "original"
 
 # Common column names containing usernames (used for hash cracking in some cases)
-COMMON_USER_COLUMNS = ('user', 'username', 'user_name', 'benutzername', 'benutzer', 'utilisateur', 'usager', 'consommateur', 'utente', 'utilizzatore', 'usufrutuario', 'korisnik', 'usuario', 'consumidor')
+COMMON_USER_COLUMNS = ("login", "user", "username", "user_name", "user_login", "benutzername", "benutzer", "utilisateur", "usager", "consommateur", "utente", "utilizzatore", "usufrutuario", "korisnik", "usuario", "consumidor", "client", "cuser")
 
 # Default delimiter in GET/POST values
 DEFAULT_GET_POST_DELIMITER = '&'
@@ -483,7 +552,7 @@ DEFAULT_COOKIE_DELIMITER = ';'
 FORCE_COOKIE_EXPIRATION_TIME = "9999999999"
 
 # Github OAuth token used for creating an automatic Issue for unhandled exceptions
-GITHUB_REPORT_OAUTH_TOKEN = "YzQzM2M2YzgzMDExN2I5ZDMyYjAzNTIzODIwZDA2MDFmMmVjODI1Ng=="
+GITHUB_REPORT_OAUTH_TOKEN = "NTMyNWNkMmZkMzRlMDZmY2JkMmY0MGI4NWI0MzVlM2Q5YmFjYWNhYQ=="
 
 # Skip unforced HashDB flush requests below the threshold number of cached items
 HASHDB_FLUSH_THRESHOLD = 32
@@ -491,11 +560,14 @@ HASHDB_FLUSH_THRESHOLD = 32
 # Number of retries for unsuccessful HashDB flush attempts
 HASHDB_FLUSH_RETRIES = 3
 
+# Number of retries for unsuccessful HashDB retrieve attempts
+HASHDB_RETRIEVE_RETRIES = 3
+
 # Number of retries for unsuccessful HashDB end transaction attempts
 HASHDB_END_TRANSACTION_RETRIES = 3
 
 # Unique milestone value used for forced deprecation of old HashDB values (e.g. when changing hash/pickle mechanism)
-HASHDB_MILESTONE_VALUE = "JHjrBugdDA"  # "".join(random.sample(string.ascii_letters, 10))
+HASHDB_MILESTONE_VALUE = "dPHoJRQYvs"  # python -c 'import random, string; print "".join(random.sample(string.ascii_letters, 10))'
 
 # Warn user of possible delay due to large page dump in full UNION query injections
 LARGE_OUTPUT_THRESHOLD = 1024 ** 2
@@ -521,14 +593,26 @@ DNS_BOUNDARIES_ALPHABET = re.sub("[a-fA-F]", "", string.ascii_letters)
 # Alphabet used for heuristic checks
 HEURISTIC_CHECK_ALPHABET = ('"', '\'', ')', '(', ',', '.')
 
-# String used for dummy XSS check of a tested parameter value
+# Minor artistic touch
-DUMMY_XSS_CHECK_APPENDIX = "<'\">"
+BANNER = re.sub(r"\[.\]", lambda _: "[\033[01;41m%s\033[01;49m]" % random.sample(HEURISTIC_CHECK_ALPHABET, 1)[0], BANNER)
+
+# String used for dummy non-SQLi (e.g. XSS) heuristic checks of a tested parameter value
+DUMMY_NON_SQLI_CHECK_APPENDIX = "<'\">"
+
+# Regular expression used for recognition of file inclusion errors
+FI_ERROR_REGEX = "(?i)[^\n]{0,100}(no such file|failed (to )?open)[^\n]{0,100}"
+
+# Length of prefix and suffix used in non-SQLI heuristic checks
+NON_SQLI_CHECK_PREFIX_SUFFIX_LENGTH = 6
 
 # Connection chunk size (processing large responses in chunks to avoid MemoryError crashes - e.g. large table dump in full UNION injections)
 MAX_CONNECTION_CHUNK_SIZE = 10 * 1024 * 1024
 
 # Maximum response total page size (trimmed if larger)
-MAX_CONNECTION_TOTAL_SIZE = 100 * 1024 * 1024
+MAX_CONNECTION_TOTAL_SIZE = 50 * 1024 * 1024
+
+# For preventing MemoryError exceptions (caused when using large sequences in difflib.SequenceMatcher)
+MAX_DIFFLIB_SEQUENCE_LENGTH = 10 * 1024 * 1024
 
 # Maximum (multi-threaded) length of entry in bisection algorithm
 MAX_BISECTION_LENGTH = 50 * 1024 * 1024
@@ -537,7 +621,7 @@ MAX_BISECTION_LENGTH = 50 * 1024 * 1024
 LARGE_CHUNK_TRIM_MARKER = "__TRIMMED_CONTENT__"
 
 # Generic SQL comment formation
-GENERIC_SQL_COMMENT = "-- "
+GENERIC_SQL_COMMENT = "-- [RANDSTR]"
 
 # Threshold value for turning back on time auto-adjustment mechanism
 VALID_TIME_CHARS_RUN_THRESHOLD = 100
@@ -546,7 +630,7 @@ VALID_TIME_CHARS_RUN_THRESHOLD = 100
 CHECK_ZERO_COLUMNS_THRESHOLD = 10
 
 # Boldify all logger messages containing these "patterns"
-BOLD_PATTERNS = ("' injectable", "might be injectable", "' is vulnerable", "is not injectable", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved")
+BOLD_PATTERNS = ("' injectable", "provided empty", "leftover chars", "might be injectable", "' is vulnerable", "is not injectable", "does not seem to be", "test failed", "test passed", "live test final result", "test shows that", "the back-end DBMS is", "created Github", "blocked by the target server", "protection is involved", "CAPTCHA")
 
 # Generic www root directory names
 GENERIC_DOC_ROOT_DIRECTORY_NAMES = ("htdocs", "httpdocs", "public", "wwwroot", "www")
@@ -558,7 +642,7 @@ MAX_HELP_OPTION_LENGTH = 18
 MAX_CONNECT_RETRIES = 100
 
 # Strings for detecting formatting errors
-FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Failed to convert", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal")
+FORMAT_EXCEPTION_STRINGS = ("Type mismatch", "Error converting", "Conversion failed", "String or binary data would be truncated", "Failed to convert", "unable to interpret text value", "Input string was not in a correct format", "System.FormatException", "java.lang.NumberFormatException", "ValueError: invalid literal", "DataTypeMismatchException", "CF_SQL_INTEGER", " for CFSQLTYPE ", "cfqueryparam cfsqltype", "InvalidParamTypeException", "Invalid parameter type", "is not of type numeric", "<cfif Not IsNumeric(", "invalid input syntax for integer", "invalid input syntax for type", "invalid number", "character to number conversion error", "unable to interpret text value", "String was not recognized as a valid", "Convert.ToInt", "cannot be converted to a ", "InvalidDataException")
 
 # Regular expression used for extracting ASP.NET view state values
 VIEWSTATE_REGEX = r'(?i)(?P<name>__VIEWSTATE[^"]*)[^>]+value="(?P<result>[^"]+)'
@@ -569,8 +653,17 @@ EVENTVALIDATION_REGEX = r'(?i)(?P<name>__EVENTVALIDATION[^"]*)[^>]+value="(?P<re
 # Number of rows to generate inside the full union test for limited output (mustn't be too large to prevent payload length problems)
 LIMITED_ROWS_TEST_NUMBER = 15
 
+# Default adapter to use for bottle server
+RESTAPI_DEFAULT_ADAPTER = "wsgiref"
+
+# Default REST-JSON API server listen address
+RESTAPI_DEFAULT_ADDRESS = "127.0.0.1"
+
+# Default REST-JSON API server listen port
+RESTAPI_DEFAULT_PORT = 8775
+
 # Format used for representing invalid unicode characters
-INVALID_UNICODE_CHAR_FORMAT = r"\?%02x"
+INVALID_UNICODE_CHAR_FORMAT = r"\x%02x"
 
 # Regular expression for XML POST data
 XML_RECOGNITION_REGEX = r"(?s)\A\s*<[^>]+>(.+>)?\s*\Z"
@@ -599,6 +692,9 @@ SUHOSIN_MAX_VALUE_LENGTH = 512
 # Minimum size of an (binary) entry before it can be considered for dumping to disk
 MIN_BINARY_DISK_DUMP_SIZE = 100
 
+# Filenames of payloads xml files (in order of loading)
+PAYLOAD_XML_FILES = ("boolean_blind.xml", "error_based.xml", "inline_query.xml", "stacked_queries.xml", "time_blind.xml", "union_query.xml")
+
 # Regular expression used for extracting form tags
 FORM_SEARCH_REGEX = r"(?si)<form(?!.+<form).+?</form>"
 
@@ -609,7 +705,7 @@ MAX_HISTORY_LENGTH = 1000
 MIN_ENCODED_LEN_CHECK = 5
 
 # Timeout in seconds in which Metasploit remote session has to be initialized
-METASPLOIT_SESSION_TIMEOUT = 300
+METASPLOIT_SESSION_TIMEOUT = 120
 
 # Reference: http://www.postgresql.org/docs/9.0/static/catalog-pg-largeobject.html
 LOBLKSIZE = 2048
@@ -630,7 +726,7 @@ BRUTE_DOC_ROOT_PREFIXES = {
 }
 
 # Suffixes used in brute force search for web server document root
-BRUTE_DOC_ROOT_SUFFIXES = ("", "html", "htdocs", "httpdocs", "php", "public", "src", "site", "build", "web", "data", "sites/all", "www/build")
+BRUTE_DOC_ROOT_SUFFIXES = ("", "html", "htdocs", "httpdocs", "php", "public", "src", "site", "build", "web", "www", "data", "sites/all", "www/build")
 
 # String used for marking target name inside used brute force web server document root
 BRUTE_DOC_ROOT_TARGET_MARK = "%TARGET%"

lib/core/shell.py

@@ -1,22 +1,43 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
 import atexit
 import os
-import rlcompleter
 
 from lib.core import readlineng as readline
-from lib.core.common import Backend
 from lib.core.data import logger
 from lib.core.data import paths
 from lib.core.enums import AUTOCOMPLETE_TYPE
 from lib.core.enums import OS
 from lib.core.settings import MAX_HISTORY_LENGTH
 
+try:
+    import rlcompleter
+
+    class CompleterNG(rlcompleter.Completer):
+        def global_matches(self, text):
+            """
+            Compute matches when text is a simple name.
+            Return a list of all names currently defined in self.namespace
+            that match.
+            """
+
+            matches = []
+            n = len(text)
+
+            for ns in (self.namespace,):
+                for word in ns:
+                    if word[:n] == text:
+                        matches.append(word)
+
+            return matches
+except:
+    readline._readline = None
+
 def readlineAvailable():
     """
     Check if the readline is available. By default
@@ -75,24 +96,6 @@ def loadHistory(completion=None):
             warnMsg = "there was a problem loading the history file '%s' (%s)" % (historyPath, msg)
             logger.warn(warnMsg)
 
-class CompleterNG(rlcompleter.Completer):
-    def global_matches(self, text):
-        """
-        Compute matches when text is a simple name.
-        Return a list of all names currently defined in self.namespace
-        that match.
-        """
-
-        matches = []
-        n = len(text)
-
-        for ns in (self.namespace,):
-            for word in ns:
-                if word[:n] == text:
-                    matches.append(word)
-
-        return matches
-
 def autoCompletion(completion=None, os=None, commands=None):
     if not readlineAvailable():
         return

lib/core/subprocessng.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/core/target.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -9,17 +9,21 @@ import codecs
 import functools
 import os
 import re
+import subprocess
+import sys
 import tempfile
 import time
 import urlparse
 
 from lib.core.common import Backend
+from lib.core.common import getSafeExString
 from lib.core.common import getUnicode
 from lib.core.common import hashDBRetrieve
 from lib.core.common import intersect
 from lib.core.common import normalizeUnicode
 from lib.core.common import openFile
 from lib.core.common import paramToDict
+from lib.core.common import randomStr
 from lib.core.common import readInput
 from lib.core.common import resetCookieJar
 from lib.core.common import urldecode
@@ -34,12 +38,12 @@ from lib.core.dump import dumper
 from lib.core.enums import HASHDB_KEYS
 from lib.core.enums import HTTP_HEADER
 from lib.core.enums import HTTPMETHOD
+from lib.core.enums import MKSTEMP_PREFIX
 from lib.core.enums import PLACE
 from lib.core.enums import POST_HINT
 from lib.core.exception import SqlmapFilePathException
 from lib.core.exception import SqlmapGenericException
 from lib.core.exception import SqlmapMissingPrivileges
-from lib.core.exception import SqlmapSyntaxException
 from lib.core.exception import SqlmapSystemException
 from lib.core.exception import SqlmapUserQuitException
 from lib.core.option import _setDBMS
@@ -66,7 +70,6 @@ from lib.core.settings import URI_INJECTABLE_REGEX
 from lib.core.settings import USER_AGENT_ALIASES
 from lib.core.settings import XML_RECOGNITION_REGEX
 from lib.utils.hashdb import HashDB
-from lib.core.xmldump import dumper as xmldumper
 from thirdparty.odict.odict import OrderedDict
 
 def _setRequestParams():
@@ -92,8 +95,8 @@ def _setRequestParams():
 
     # Perform checks on POST parameters
     if conf.method == HTTPMETHOD.POST and conf.data is None:
-        errMsg = "HTTP POST method depends on HTTP data value to be posted"
+        logger.warn("detected empty POST body")
-        raise SqlmapSyntaxException(errMsg)
+        conf.data = ""
 
     if conf.data is not None:
         conf.method = HTTPMETHOD.POST if not conf.method or conf.method == HTTPMETHOD.GET else conf.method
@@ -117,23 +120,26 @@ def _setRequestParams():
         if kb.processUserMarks is None and CUSTOM_INJECTION_MARK_CHAR in conf.data:
             message = "custom injection marking character ('%s') found in option " % CUSTOM_INJECTION_MARK_CHAR
             message += "'--data'. Do you want to process it? [Y/n/q] "
-            test = readInput(message, default="Y")
+            choice = readInput(message, default='Y')
-            if test and test[0] in ("q", "Q"):
+
+            if choice == 'Q':
                 raise SqlmapUserQuitException
             else:
-                kb.processUserMarks = not test or test[0] not in ("n", "N")
+                kb.processUserMarks = choice == 'Y'
 
                 if kb.processUserMarks:
                     kb.testOnlyCustom = True
 
-        if not (kb.processUserMarks and CUSTOM_INJECTION_MARK_CHAR in conf.data):
+        if re.search(JSON_RECOGNITION_REGEX, conf.data):
-            if re.search(JSON_RECOGNITION_REGEX, conf.data):
+            message = "JSON data found in %s data. " % conf.method
-                message = "JSON data found in %s data. " % conf.method
+            message += "Do you want to process it? [Y/n/q] "
-                message += "Do you want to process it? [Y/n/q] "
+            choice = readInput(message, default='Y')
-                test = readInput(message, default="Y")
+
-                if test and test[0] in ("q", "Q"):
+            if choice == 'Q':
-                    raise SqlmapUserQuitException
+                raise SqlmapUserQuitException
-                elif test[0] not in ("n", "N"):
+            elif choice == 'Y':
+                if not (kb.processUserMarks and CUSTOM_INJECTION_MARK_CHAR in conf.data):
+                    conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data)
                     conf.data = conf.data.replace(CUSTOM_INJECTION_MARK_CHAR, ASTERISK_MARKER)
                     conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*"[^"]+)"', functools.partial(process, repl=r'\g<1>%s"' % CUSTOM_INJECTION_MARK_CHAR), conf.data)
                     conf.data = re.sub(r'("(?P<name>[^"]+)"\s*:\s*)(-?\d[\d\.]*\b)', functools.partial(process, repl=r'\g<0>%s' % CUSTOM_INJECTION_MARK_CHAR), conf.data)
@@ -143,52 +149,68 @@ def _setRequestParams():
                         _ = re.sub(r'("[^"]+)"', '\g<1>%s"' % CUSTOM_INJECTION_MARK_CHAR, _)
                         _ = re.sub(r'(\A|,|\s+)(-?\d[\d\.]*\b)', '\g<0>%s' % CUSTOM_INJECTION_MARK_CHAR, _)
                         conf.data = conf.data.replace(match.group(0), match.group(0).replace(match.group(2), _))
-                    kb.postHint = POST_HINT.JSON
 
-            elif re.search(JSON_LIKE_RECOGNITION_REGEX, conf.data):
+                kb.postHint = POST_HINT.JSON
-                message = "JSON-like data found in %s data. " % conf.method
+
-                message += "Do you want to process it? [Y/n/q] "
+        elif re.search(JSON_LIKE_RECOGNITION_REGEX, conf.data):
-                test = readInput(message, default="Y")
+            message = "JSON-like data found in %s data. " % conf.method
-                if test and test[0] in ("q", "Q"):
+            message += "Do you want to process it? [Y/n/q] "
-                    raise SqlmapUserQuitException
+            choice = readInput(message, default='Y').upper()
-                elif test[0] not in ("n", "N"):
+
+            if choice == 'Q':
+                raise SqlmapUserQuitException
+            elif choice == 'Y':
+                if not (kb.processUserMarks and CUSTOM_INJECTION_MARK_CHAR in conf.data):
+                    conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data)
                     conf.data = conf.data.replace(CUSTOM_INJECTION_MARK_CHAR, ASTERISK_MARKER)
                     conf.data = re.sub(r"('(?P<name>[^']+)'\s*:\s*'[^']+)'", functools.partial(process, repl=r"\g<1>%s'" % CUSTOM_INJECTION_MARK_CHAR), conf.data)
                     conf.data = re.sub(r"('(?P<name>[^']+)'\s*:\s*)(-?\d[\d\.]*\b)", functools.partial(process, repl=r"\g<0>%s" % CUSTOM_INJECTION_MARK_CHAR), conf.data)
-                    kb.postHint = POST_HINT.JSON_LIKE
 
-            elif re.search(ARRAY_LIKE_RECOGNITION_REGEX, conf.data):
+                kb.postHint = POST_HINT.JSON_LIKE
-                message = "Array-like data found in %s data. " % conf.method
+
-                message += "Do you want to process it? [Y/n/q] "
+        elif re.search(ARRAY_LIKE_RECOGNITION_REGEX, conf.data):
-                test = readInput(message, default="Y")
+            message = "Array-like data found in %s data. " % conf.method
-                if test and test[0] in ("q", "Q"):
+            message += "Do you want to process it? [Y/n/q] "
-                    raise SqlmapUserQuitException
+            choice = readInput(message, default='Y').upper()
-                elif test[0] not in ("n", "N"):
+
+            if choice == 'Q':
+                raise SqlmapUserQuitException
+            elif choice == 'Y':
+                if not (kb.processUserMarks and CUSTOM_INJECTION_MARK_CHAR in conf.data):
                     conf.data = conf.data.replace(CUSTOM_INJECTION_MARK_CHAR, ASTERISK_MARKER)
                     conf.data = re.sub(r"(=[^%s]+)" % DEFAULT_GET_POST_DELIMITER, r"\g<1>%s" % CUSTOM_INJECTION_MARK_CHAR, conf.data)
-                    kb.postHint = POST_HINT.ARRAY_LIKE
 
-            elif re.search(XML_RECOGNITION_REGEX, conf.data):
+                kb.postHint = POST_HINT.ARRAY_LIKE
-                message = "SOAP/XML data found in %s data. " % conf.method
+
-                message += "Do you want to process it? [Y/n/q] "
+        elif re.search(XML_RECOGNITION_REGEX, conf.data):
-                test = readInput(message, default="Y")
+            message = "SOAP/XML data found in %s data. " % conf.method
-                if test and test[0] in ("q", "Q"):
+            message += "Do you want to process it? [Y/n/q] "
-                    raise SqlmapUserQuitException
+            choice = readInput(message, default='Y').upper()
-                elif test[0] not in ("n", "N"):
+
+            if choice == 'Q':
+                raise SqlmapUserQuitException
+            elif choice == 'Y':
+                if not (kb.processUserMarks and CUSTOM_INJECTION_MARK_CHAR in conf.data):
+                    conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data)
                     conf.data = conf.data.replace(CUSTOM_INJECTION_MARK_CHAR, ASTERISK_MARKER)
                     conf.data = re.sub(r"(<(?P<name>[^>]+)( [^<]*)?>)([^<]+)(</\2)", functools.partial(process, repl=r"\g<1>\g<4>%s\g<5>" % CUSTOM_INJECTION_MARK_CHAR), conf.data)
-                    kb.postHint = POST_HINT.SOAP if "soap" in conf.data.lower() else POST_HINT.XML
 
-            elif re.search(MULTIPART_RECOGNITION_REGEX, conf.data):
+                kb.postHint = POST_HINT.SOAP if "soap" in conf.data.lower() else POST_HINT.XML
-                message = "Multipart-like data found in %s data. " % conf.method
+
-                message += "Do you want to process it? [Y/n/q] "
+        elif re.search(MULTIPART_RECOGNITION_REGEX, conf.data):
-                test = readInput(message, default="Y")
+            message = "Multipart-like data found in %s data. " % conf.method
-                if test and test[0] in ("q", "Q"):
+            message += "Do you want to process it? [Y/n/q] "
-                    raise SqlmapUserQuitException
+            choice = readInput(message, default='Y').upper()
-                elif test[0] not in ("n", "N"):
+
+            if choice == 'Q':
+                raise SqlmapUserQuitException
+            elif choice == 'Y':
+                if not (kb.processUserMarks and CUSTOM_INJECTION_MARK_CHAR in conf.data):
+                    conf.data = getattr(conf.data, UNENCODED_ORIGINAL_VALUE, conf.data)
                     conf.data = conf.data.replace(CUSTOM_INJECTION_MARK_CHAR, ASTERISK_MARKER)
                     conf.data = re.sub(r"(?si)((Content-Disposition[^\n]+?name\s*=\s*[\"'](?P<name>[^\n]+?)[\"']).+?)(((\r)?\n)+--)", functools.partial(process, repl=r"\g<1>%s\g<4>" % CUSTOM_INJECTION_MARK_CHAR), conf.data)
-                    kb.postHint = POST_HINT.MULTIPART
+
+                kb.postHint = POST_HINT.MULTIPART
 
         if not kb.postHint:
             if CUSTOM_INJECTION_MARK_CHAR in conf.data:  # later processed
@@ -210,20 +232,20 @@ def _setRequestParams():
 
     if re.search(URI_INJECTABLE_REGEX, conf.url, re.I) and not any(place in conf.parameters for place in (PLACE.GET, PLACE.POST)) and not kb.postHint and not CUSTOM_INJECTION_MARK_CHAR in (conf.data or "") and conf.url.startswith("http"):
         warnMsg = "you've provided target URL without any GET "
-        warnMsg += "parameters (e.g. www.site.com/article.php?id=1) "
+        warnMsg += "parameters (e.g. 'http://www.site.com/article.php?id=1') "
         warnMsg += "and without providing any POST parameters "
-        warnMsg += "through --data option"
+        warnMsg += "through option '--data'"
         logger.warn(warnMsg)
 
         message = "do you want to try URI injections "
         message += "in the target URL itself? [Y/n/q] "
-        test = readInput(message, default="Y")
+        choice = readInput(message, default='Y').upper()
 
-        if not test or test[0] not in ("n", "N"):
+        if choice == 'Q':
+            raise SqlmapUserQuitException
+        elif choice == 'Y':
             conf.url = "%s%s" % (conf.url, CUSTOM_INJECTION_MARK_CHAR)
             kb.processUserMarks = True
-        elif test[0] in ("q", "Q"):
-            raise SqlmapUserQuitException
 
     for place, value in ((PLACE.URI, conf.url), (PLACE.CUSTOM_POST, conf.data), (PLACE.CUSTOM_HEADER, str(conf.httpHeaders))):
         _ = re.sub(PROBLEMATIC_CUSTOM_INJECTION_PATTERNS, "", value or "") if place == PLACE.CUSTOM_HEADER else value or ""
@@ -232,11 +254,12 @@ def _setRequestParams():
                 lut = {PLACE.URI: '-u', PLACE.CUSTOM_POST: '--data', PLACE.CUSTOM_HEADER: '--headers/--user-agent/--referer/--cookie'}
                 message = "custom injection marking character ('%s') found in option " % CUSTOM_INJECTION_MARK_CHAR
                 message += "'%s'. Do you want to process it? [Y/n/q] " % lut[place]
-                test = readInput(message, default="Y")
+                choice = readInput(message, default='Y').upper()
-                if test and test[0] in ("q", "Q"):
+
+                if choice == 'Q':
                     raise SqlmapUserQuitException
                 else:
-                    kb.processUserMarks = not test or test[0] not in ("n", "N")
+                    kb.processUserMarks = choice == 'Y'
 
                     if kb.processUserMarks:
                         kb.testOnlyCustom = True
@@ -315,39 +338,46 @@ def _setRequestParams():
 
     # Perform checks on header values
     if conf.httpHeaders:
-        for httpHeader, headerValue in conf.httpHeaders:
+        for httpHeader, headerValue in list(conf.httpHeaders):
             # Url encoding of the header values should be avoided
             # Reference: http://stackoverflow.com/questions/5085904/is-ok-to-urlencode-the-value-in-headerlocation-value
 
-            httpHeader = httpHeader.title()
+            if httpHeader.title() == HTTP_HEADER.USER_AGENT:
-
-            if httpHeader == HTTP_HEADER.USER_AGENT:
                 conf.parameters[PLACE.USER_AGENT] = urldecode(headerValue)
 
-                condition = any((not conf.testParameter, intersect(conf.testParameter, USER_AGENT_ALIASES)))
+                condition = any((not conf.testParameter, intersect(conf.testParameter, USER_AGENT_ALIASES, True)))
 
                 if condition:
                     conf.paramDict[PLACE.USER_AGENT] = {PLACE.USER_AGENT: headerValue}
                     testableParameters = True
 
-            elif httpHeader == HTTP_HEADER.REFERER:
+            elif httpHeader.title() == HTTP_HEADER.REFERER:
                 conf.parameters[PLACE.REFERER] = urldecode(headerValue)
 
-                condition = any((not conf.testParameter, intersect(conf.testParameter, REFERER_ALIASES)))
+                condition = any((not conf.testParameter, intersect(conf.testParameter, REFERER_ALIASES, True)))
 
                 if condition:
                     conf.paramDict[PLACE.REFERER] = {PLACE.REFERER: headerValue}
                     testableParameters = True
 
-            elif httpHeader == HTTP_HEADER.HOST:
+            elif httpHeader.title() == HTTP_HEADER.HOST:
                 conf.parameters[PLACE.HOST] = urldecode(headerValue)
 
-                condition = any((not conf.testParameter, intersect(conf.testParameter, HOST_ALIASES)))
+                condition = any((not conf.testParameter, intersect(conf.testParameter, HOST_ALIASES, True)))
 
                 if condition:
                     conf.paramDict[PLACE.HOST] = {PLACE.HOST: headerValue}
                     testableParameters = True
 
+            else:
+                condition = intersect(conf.testParameter, [httpHeader], True)
+
+                if condition:
+                    conf.parameters[PLACE.CUSTOM_HEADER] = str(conf.httpHeaders)
+                    conf.paramDict[PLACE.CUSTOM_HEADER] = {httpHeader: "%s,%s%s" % (httpHeader, headerValue, CUSTOM_INJECTION_MARK_CHAR)}
+                    conf.httpHeaders = [(header, value.replace(CUSTOM_INJECTION_MARK_CHAR, "")) for header, value in conf.httpHeaders]
+                    testableParameters = True
+
     if not conf.parameters:
         errMsg = "you did not provide any GET, POST and Cookie "
         errMsg += "parameter, neither an User-Agent, Referer or Host header value"
@@ -359,7 +389,7 @@ def _setRequestParams():
         raise SqlmapGenericException(errMsg)
 
     if conf.csrfToken:
-        if not any(conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))) and not conf.csrfToken in set(_[0].lower() for _ in conf.httpHeaders) and not conf.csrfToken in conf.paramDict.get(PLACE.COOKIE, {}):
+        if not any(conf.csrfToken in _ for _ in (conf.paramDict.get(PLACE.GET, {}), conf.paramDict.get(PLACE.POST, {}))) and not re.search(r"\b%s\b" % re.escape(conf.csrfToken), conf.data or "") and not conf.csrfToken in set(_[0].lower() for _ in conf.httpHeaders) and not conf.csrfToken in conf.paramDict.get(PLACE.COOKIE, {}):
             errMsg = "anti-CSRF token parameter '%s' not " % conf.csrfToken
             errMsg += "found in provided GET, POST, Cookie or header values"
             raise SqlmapGenericException(errMsg)
@@ -369,9 +399,9 @@ def _setRequestParams():
                 if any(parameter.lower().count(_) for _ in CSRF_TOKEN_PARAMETER_INFIXES):
                     message = "%s parameter '%s' appears to hold anti-CSRF token. " % (place, parameter)
                     message += "Do you want sqlmap to automatically update it in further requests? [y/N] "
-                    test = readInput(message, default="N")
+
-                    if test and test[0] in ("y", "Y"):
+                    if readInput(message, default='N', boolean=True):
-                        conf.csrfToken = parameter
+                        conf.csrfToken = getUnicode(parameter)
                     break
 
 def _setHashDB():
@@ -399,12 +429,18 @@ def _resumeHashDBValues():
     """
 
     kb.absFilePaths = hashDBRetrieve(HASHDB_KEYS.KB_ABS_FILE_PATHS, True) or kb.absFilePaths
-    kb.chars = hashDBRetrieve(HASHDB_KEYS.KB_CHARS, True) or kb.chars
-    kb.dynamicMarkings = hashDBRetrieve(HASHDB_KEYS.KB_DYNAMIC_MARKINGS, True) or kb.dynamicMarkings
     kb.brute.tables = hashDBRetrieve(HASHDB_KEYS.KB_BRUTE_TABLES, True) or kb.brute.tables
     kb.brute.columns = hashDBRetrieve(HASHDB_KEYS.KB_BRUTE_COLUMNS, True) or kb.brute.columns
+    kb.chars = hashDBRetrieve(HASHDB_KEYS.KB_CHARS, True) or kb.chars
+    kb.dynamicMarkings = hashDBRetrieve(HASHDB_KEYS.KB_DYNAMIC_MARKINGS, True) or kb.dynamicMarkings
     kb.xpCmdshellAvailable = hashDBRetrieve(HASHDB_KEYS.KB_XP_CMDSHELL_AVAILABLE) or kb.xpCmdshellAvailable
 
+    kb.errorChunkLength = hashDBRetrieve(HASHDB_KEYS.KB_ERROR_CHUNK_LENGTH)
+    if kb.errorChunkLength and kb.errorChunkLength.isdigit():
+        kb.errorChunkLength = int(kb.errorChunkLength)
+    else:
+        kb.errorChunkLength = None
+
     conf.tmpPath = conf.tmpPath or hashDBRetrieve(HASHDB_KEYS.CONF_TMP_PATH)
 
     for injection in hashDBRetrieve(HASHDB_KEYS.KB_INJECTIONS, True) or []:
@@ -413,7 +449,7 @@ def _resumeHashDBValues():
 
             if not conf.tech or intersect(conf.tech, injection.data.keys()):
                 if intersect(conf.tech, injection.data.keys()):
-                    injection.data = dict(filter(lambda (key, item): key in conf.tech, injection.data.items()))
+                    injection.data = dict(_ for _ in injection.data.items() if _[0] in conf.tech)
 
                 if injection not in kb.injections:
                     kb.injections.append(injection)
@@ -434,7 +470,7 @@ def _resumeDBMS():
     dbms = value.lower()
     dbmsVersion = [UNKNOWN_DBMS_VERSION]
     _ = "(%s)" % ("|".join([alias for alias in SUPPORTED_DBMS]))
-    _ = re.search("%s ([\d\.]+)" % _, dbms, re.I)
+    _ = re.search(r"\A%s (.*)" % _, dbms, re.I)
 
     if _:
         dbms = _.group(1).lower()
@@ -453,9 +489,8 @@ def _resumeDBMS():
             message += "sqlmap assumes the back-end DBMS is '%s'. " % dbms
             message += "Do you really want to force the back-end "
             message += "DBMS value? [y/N] "
-            test = readInput(message, default="N")
 
-            if not test or test[0] in ("n", "N"):
+            if not readInput(message, default='N', boolean=True):
                 conf.dbms = None
                 Backend.setDbms(dbms)
                 Backend.setVersionList(dbmsVersion)
@@ -489,9 +524,8 @@ def _resumeOS():
             message += "operating system is %s. " % os
             message += "Do you really want to force the back-end DBMS "
             message += "OS value? [y/N] "
-            test = readInput(message, default="N")
 
-            if not test or test[0] in ("n", "N"):
+            if not readInput(message, default='N', boolean=True):
                 conf.os = os
         else:
             conf.os = os
@@ -514,7 +548,8 @@ def _setResultsFile():
         except (OSError, IOError), ex:
             try:
                 warnMsg = "unable to create results file '%s' ('%s'). " % (conf.resultsFilename, getUnicode(ex))
-                conf.resultsFilename = tempfile.mkstemp(prefix="sqlmapresults-", suffix=".csv")[1]
+                handle, conf.resultsFilename = tempfile.mkstemp(prefix=MKSTEMP_PREFIX.RESULTS, suffix=".csv")
+                os.close(handle)
                 conf.resultsFP = openFile(conf.resultsFilename, "w+", UNICODE_ENCODING, buffering=0)
                 warnMsg += "Using temporary file '%s' instead" % conf.resultsFilename
                 logger.warn(warnMsg)
@@ -525,7 +560,7 @@ def _setResultsFile():
                 errMsg += "create temporary files and/or directories"
                 raise SqlmapSystemException(errMsg)
 
-        conf.resultsFP.writelines("Target URL,Place,Parameter,Techniques%s" % os.linesep)
+        conf.resultsFP.writelines("Target URL,Place,Parameter,Technique(s),Note(s)%s" % os.linesep)
 
         logger.info("using '%s' as the CSV results file in multiple targets mode" % conf.resultsFilename)
 
@@ -574,11 +609,7 @@ def _createDumpDir():
             conf.dumpPath = tempDir
 
 def _configureDumper():
-    if hasattr(conf, 'xmlFile') and conf.xmlFile:
+    conf.dumper = dumper
-        conf.dumper = xmldumper
-    else:
-        conf.dumper = dumper
-
     conf.dumper.setOutputFile()
 
 def _createTargetDirs():
@@ -586,28 +617,33 @@ def _createTargetDirs():
     Create the output directory.
     """
 
-    if not os.path.isdir(paths.SQLMAP_OUTPUT_PATH):
+    try:
-        try:
+        if not os.path.isdir(paths.SQLMAP_OUTPUT_PATH):
-            if not os.path.isdir(paths.SQLMAP_OUTPUT_PATH):
+            os.makedirs(paths.SQLMAP_OUTPUT_PATH, 0755)
-                os.makedirs(paths.SQLMAP_OUTPUT_PATH, 0755)
+
+        _ = os.path.join(paths.SQLMAP_OUTPUT_PATH, randomStr())
+        open(_, "w+b").close()
+        os.remove(_)
+
+        if conf.outputDir:
             warnMsg = "using '%s' as the output directory" % paths.SQLMAP_OUTPUT_PATH
             logger.warn(warnMsg)
-        except (OSError, IOError), ex:
+    except (OSError, IOError), ex:
-            try:
+        try:
-                tempDir = tempfile.mkdtemp(prefix="sqlmapoutput")
+            tempDir = tempfile.mkdtemp(prefix="sqlmapoutput")
-            except Exception, _:
+        except Exception, _:
-                errMsg = "unable to write to the temporary directory ('%s'). " % _
+            errMsg = "unable to write to the temporary directory ('%s'). " % _
-                errMsg += "Please make sure that your disk is not full and "
+            errMsg += "Please make sure that your disk is not full and "
-                errMsg += "that you have sufficient write permissions to "
+            errMsg += "that you have sufficient write permissions to "
-                errMsg += "create temporary files and/or directories"
+            errMsg += "create temporary files and/or directories"
-                raise SqlmapSystemException(errMsg)
+            raise SqlmapSystemException(errMsg)
 
-            warnMsg = "unable to create regular output directory "
+        warnMsg = "unable to %s output directory " % ("create" if not os.path.isdir(paths.SQLMAP_OUTPUT_PATH) else "write to the")
-            warnMsg += "'%s' (%s). " % (paths.SQLMAP_OUTPUT_PATH, getUnicode(ex))
+        warnMsg += "'%s' (%s). " % (paths.SQLMAP_OUTPUT_PATH, getUnicode(ex))
-            warnMsg += "Using temporary directory '%s' instead" % getUnicode(tempDir)
+        warnMsg += "Using temporary directory '%s' instead" % getUnicode(tempDir)
-            logger.warn(warnMsg)
+        logger.warn(warnMsg)
 
-            paths.SQLMAP_OUTPUT_PATH = tempDir
+        paths.SQLMAP_OUTPUT_PATH = tempDir
 
     conf.outputPath = os.path.join(getUnicode(paths.SQLMAP_OUTPUT_PATH), normalizeUnicode(getUnicode(conf.hostname)))
 
@@ -635,6 +671,7 @@ def _createTargetDirs():
         with codecs.open(os.path.join(conf.outputPath, "target.txt"), "w+", UNICODE_ENCODING) as f:
             f.write(kb.originalUrls.get(conf.url) or conf.url or conf.hostname)
             f.write(" (%s)" % (HTTPMETHOD.POST if conf.data else HTTPMETHOD.GET))
+            f.write("  # %s" % getUnicode(subprocess.list2cmdline(sys.argv), encoding=sys.stdin.encoding))
             if conf.data:
                 f.write("\n\n%s" % getUnicode(conf.data))
     except IOError, ex:
@@ -642,7 +679,7 @@ def _createTargetDirs():
             errMsg = "you don't have enough permissions "
         else:
             errMsg = "something went wrong while trying "
-        errMsg += "to write to the output directory '%s' (%s)" % (paths.SQLMAP_OUTPUT_PATH, ex)
+        errMsg += "to write to the output directory '%s' (%s)" % (paths.SQLMAP_OUTPUT_PATH, getSafeExString(ex))
 
         raise SqlmapMissingPrivileges(errMsg)
 
@@ -683,10 +720,13 @@ def initTargetEnv():
         class _(unicode):
             pass
 
+        kb.postUrlEncode = True
+
         for key, value in conf.httpHeaders:
             if key.upper() == HTTP_HEADER.CONTENT_TYPE.upper():
                 kb.postUrlEncode = "urlencoded" in value
                 break
+
         if kb.postUrlEncode:
             original = conf.data
             conf.data = _(urldecode(conf.data))

lib/core/testing.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -25,6 +25,7 @@ from lib.core.common import readXmlFile
 from lib.core.data import conf
 from lib.core.data import logger
 from lib.core.data import paths
+from lib.core.enums import MKSTEMP_PREFIX
 from lib.core.exception import SqlmapBaseException
 from lib.core.exception import SqlmapNotVulnerableException
 from lib.core.log import LOGGER_HANDLER
@@ -40,6 +41,8 @@ class Failures(object):
     failedParseOn = None
     failedTraceBack = None
 
+_failures = Failures()
+
 def smokeTest():
     """
     Runs the basic smoke testing of a program
@@ -52,16 +55,17 @@ def smokeTest():
         if any(_ in root for _ in ("thirdparty", "extra")):
             continue
 
-        for ifile in files:
+        for filename in files:
-            length += 1
+            if os.path.splitext(filename)[1].lower() == ".py" and filename != "__init__.py":
+                length += 1
 
     for root, _, files in os.walk(paths.SQLMAP_ROOT_PATH):
         if any(_ in root for _ in ("thirdparty", "extra")):
             continue
 
-        for ifile in files:
+        for filename in files:
-            if os.path.splitext(ifile)[1].lower() == ".py" and ifile != "__init__.py":
+            if os.path.splitext(filename)[1].lower() == ".py" and filename != "__init__.py":
-                path = os.path.join(root, os.path.splitext(ifile)[0])
+                path = os.path.join(root, os.path.splitext(filename)[0])
                 path = path.replace(paths.SQLMAP_ROOT_PATH, '.')
                 path = path.replace(os.sep, '.').lstrip('.')
                 try:
@@ -70,7 +74,7 @@ def smokeTest():
                 except Exception, msg:
                     retVal = False
                     dataToStdout("\r")
-                    errMsg = "smoke test failed at importing module '%s' (%s):\n%s" % (path, os.path.join(root, ifile), msg)
+                    errMsg = "smoke test failed at importing module '%s' (%s):\n%s" % (path, os.path.join(root, filename), msg)
                     logger.error(errMsg)
                 else:
                     # Run doc tests
@@ -79,9 +83,9 @@ def smokeTest():
                     if failure_count > 0:
                         retVal = False
 
-            count += 1
+                count += 1
-            status = '%d/%d (%d%%) ' % (count, length, round(100.0 * count / length))
+                status = '%d/%d (%d%%) ' % (count, length, round(100.0 * count / length))
-            dataToStdout("\r[%s] [INFO] complete: %s" % (time.strftime("%X"), status))
+                dataToStdout("\r[%s] [INFO] complete: %s" % (time.strftime("%X"), status))
 
     clearConsoleLine()
     if retVal:
@@ -191,11 +195,11 @@ def liveTest():
         else:
             errMsg = "test failed"
 
-            if Failures.failedItems:
+            if _failures.failedItems:
-                errMsg += " at parsing items: %s" % ", ".join(i for i in Failures.failedItems)
+                errMsg += " at parsing items: %s" % ", ".join(i for i in _failures.failedItems)
 
             errMsg += " - scan folder: %s" % paths.SQLMAP_OUTPUT_PATH
-            errMsg += " - traceback: %s" % bool(Failures.failedTraceBack)
+            errMsg += " - traceback: %s" % bool(_failures.failedTraceBack)
 
             if not vulnerable:
                 errMsg += " - SQL injection not detected"
@@ -203,14 +207,14 @@ def liveTest():
             logger.error(errMsg)
             test_case_fd.write("%s\n" % errMsg)
 
-            if Failures.failedParseOn:
+            if _failures.failedParseOn:
                 console_output_fd = codecs.open(os.path.join(paths.SQLMAP_OUTPUT_PATH, "console_output"), "wb", UNICODE_ENCODING)
-                console_output_fd.write(Failures.failedParseOn)
+                console_output_fd.write(_failures.failedParseOn)
                 console_output_fd.close()
 
-            if Failures.failedTraceBack:
+            if _failures.failedTraceBack:
                 traceback_fd = codecs.open(os.path.join(paths.SQLMAP_OUTPUT_PATH, "traceback"), "wb", UNICODE_ENCODING)
-                traceback_fd.write(Failures.failedTraceBack)
+                traceback_fd.write(_failures.failedTraceBack)
                 traceback_fd.close()
 
             beep()
@@ -231,11 +235,11 @@ def liveTest():
     return retVal
 
 def initCase(switches, count):
-    Failures.failedItems = []
+    _failures.failedItems = []
-    Failures.failedParseOn = None
+    _failures.failedParseOn = None
-    Failures.failedTraceBack = None
+    _failures.failedTraceBack = None
 
-    paths.SQLMAP_OUTPUT_PATH = tempfile.mkdtemp(prefix="sqlmaptest-%d-" % count)
+    paths.SQLMAP_OUTPUT_PATH = tempfile.mkdtemp(prefix="%s%d-" % (MKSTEMP_PREFIX.TESTING, count))
     paths.SQLMAP_DUMP_PATH = os.path.join(paths.SQLMAP_OUTPUT_PATH, "%s", "dump")
     paths.SQLMAP_FILES_PATH = os.path.join(paths.SQLMAP_OUTPUT_PATH, "%s", "files")
 
@@ -277,10 +281,10 @@ def runCase(parse):
         LOGGER_HANDLER.stream = sys.stdout = sys.__stdout__
 
     if unhandled_exception:
-        Failures.failedTraceBack = "unhandled exception: %s" % str(traceback.format_exc())
+        _failures.failedTraceBack = "unhandled exception: %s" % str(traceback.format_exc())
         retVal = None
     elif handled_exception:
-        Failures.failedTraceBack = "handled exception: %s" % str(traceback.format_exc())
+        _failures.failedTraceBack = "handled exception: %s" % str(traceback.format_exc())
         retVal = None
     elif result is False:  # this means no SQL injection has been detected - if None, ignore
         retVal = False
@@ -297,17 +301,17 @@ def runCase(parse):
             if item.startswith("r'") and item.endswith("'"):
                 if not re.search(item[2:-1], parse_on, re.DOTALL):
                     retVal = None
-                    Failures.failedItems.append(item)
+                    _failures.failedItems.append(item)
 
             elif item not in parse_on:
                 retVal = None
-                Failures.failedItems.append(item)
+                _failures.failedItems.append(item)
 
-        if Failures.failedItems:
+        if _failures.failedItems:
-            Failures.failedParseOn = console
+            _failures.failedParseOn = console
 
     elif retVal is False:
-        Failures.failedParseOn = console
+        _failures.failedParseOn = console
 
     return retVal
 

lib/core/threads.py

@@ -1,17 +1,16 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
 import difflib
+import random
 import threading
 import time
 import traceback
 
-from thread import error as threadError
-
 from lib.core.data import conf
 from lib.core.data import kb
 from lib.core.data import logger
@@ -19,6 +18,7 @@ from lib.core.datatype import AttribDict
 from lib.core.enums import PAYLOAD
 from lib.core.exception import SqlmapConnectionException
 from lib.core.exception import SqlmapThreadException
+from lib.core.exception import SqlmapUserQuitException
 from lib.core.exception import SqlmapValueException
 from lib.core.settings import MAX_NUMBER_OF_THREADS
 from lib.core.settings import PYVERSION
@@ -38,22 +38,30 @@ class _ThreadData(threading.local):
         Resets thread data model
         """
 
+        self.requestCollector = None
+
         self.disableStdOut = False
         self.hashDBCursor = None
         self.inTransaction = False
+        self.lastCode = None
         self.lastComparisonPage = None
         self.lastComparisonHeaders = None
+        self.lastComparisonCode = None
+        self.lastComparisonRatio = None
         self.lastErrorPage = None
         self.lastHTTPError = None
         self.lastRedirectMsg = None
         self.lastQueryDuration = 0
+        self.lastPage = None
         self.lastRequestMsg = None
         self.lastRequestUID = 0
         self.lastRedirectURL = None
+        self.random = random.WichmannHill()
         self.resumed = False
         self.retriesCount = 0
         self.seqMatcher = difflib.SequenceMatcher(None)
         self.shared = shared
+        self.validationRun = 0
         self.valueStack = []
 
 ThreadData = _ThreadData()
@@ -61,7 +69,7 @@ ThreadData = _ThreadData()
 def getCurrentThreadUID():
     return hash(threading.currentThread())
 
-def readInput(message, default=None):
+def readInput(message, default=None, checkBatch=True, boolean=False):
     # It will be overwritten by original from lib.core.common
     pass
 
@@ -81,16 +89,16 @@ def getCurrentThreadName():
 
     return threading.current_thread().getName()
 
-def exceptionHandledFunction(threadFunction):
+def exceptionHandledFunction(threadFunction, silent=False):
     try:
         threadFunction()
     except KeyboardInterrupt:
         kb.threadContinue = False
         kb.threadException = True
         raise
-    except Exception, errMsg:
+    except Exception, ex:
-        # thread is just going to be silently killed
+        if not silent:
-        logger.error("thread %s: %s" % (threading.currentThread().getName(), errMsg))
+            logger.error("thread %s: %s" % (threading.currentThread().getName(), ex.message))
 
 def setDaemon(thread):
     # Reference: http://stackoverflow.com/questions/190010/daemon-threads-explanation
@@ -144,8 +152,8 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
 
             try:
                 thread.start()
-            except threadError, errMsg:
+            except Exception, ex:
-                errMsg = "error occurred while starting new thread ('%s')" % errMsg
+                errMsg = "error occurred while starting new thread ('%s')" % ex.message
                 logger.critical(errMsg)
                 break
 
@@ -160,13 +168,13 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
                     alive = True
                     time.sleep(0.1)
 
-    except KeyboardInterrupt:
+    except (KeyboardInterrupt, SqlmapUserQuitException), ex:
         print
         kb.threadContinue = False
         kb.threadException = True
 
         if numThreads > 1:
-            logger.info("waiting for threads to finish (Ctrl+C was pressed)")
+            logger.info("waiting for threads to finish%s" % (" (Ctrl+C was pressed)" if isinstance(ex, KeyboardInterrupt) else ""))
         try:
             while (threading.activeCount() > 1):
                 pass
@@ -177,10 +185,10 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
         if forwardException:
             raise
 
-    except (SqlmapConnectionException, SqlmapValueException), errMsg:
+    except (SqlmapConnectionException, SqlmapValueException), ex:
         print
         kb.threadException = True
-        logger.error("thread %s: %s" % (threading.currentThread().getName(), errMsg))
+        logger.error("thread %s: %s" % (threading.currentThread().getName(), ex.message))
 
     except:
         from lib.core.common import unhandledExceptionMessage
@@ -198,8 +206,11 @@ def runThreads(numThreads, threadFunction, cleanupFunction=None, forwardExceptio
         kb.threadException = False
 
         for lock in kb.locks.values():
-            if lock.locked_lock():
+            if lock.locked():
-                lock.release()
+                try:
+                    lock.release()
+                except:
+                    pass
 
         if conf.get("hashDB"):
             conf.hashDB.flush(True)

lib/core/unescaper.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/core/update.py

@@ -1,18 +1,18 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
+import locale
 import os
 import re
+import subprocess
 import time
 
-from subprocess import PIPE
-from subprocess import Popen as execute
-
 from lib.core.common import dataToStdout
+from lib.core.common import getSafeExString
 from lib.core.common import pollProcess
 from lib.core.data import conf
 from lib.core.data import logger
@@ -26,11 +26,10 @@ def update():
         return
 
     success = False
-    rootDir = paths.SQLMAP_ROOT_PATH
 
-    if not os.path.exists(os.path.join(rootDir, ".git")):
+    if not os.path.exists(os.path.join(paths.SQLMAP_ROOT_PATH, ".git")):
         errMsg = "not a git repository. Please checkout the 'sqlmapproject/sqlmap' repository "
-        errMsg += "from GitHub (e.g. git clone https://github.com/sqlmapproject/sqlmap.git sqlmap-dev)"
+        errMsg += "from GitHub (e.g. 'git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap')"
         logger.error(errMsg)
     else:
         infoMsg = "updating sqlmap to the latest development version from the "
@@ -41,17 +40,25 @@ def update():
         logger.debug(debugMsg)
 
         dataToStdout("\r[%s] [INFO] update in progress " % time.strftime("%X"))
-        process = execute("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=PIPE, stderr=PIPE)
+
-        pollProcess(process, True)
+        try:
-        stdout, stderr = process.communicate()
+            process = subprocess.Popen("git checkout . && git pull %s HEAD" % GIT_REPOSITORY, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=paths.SQLMAP_ROOT_PATH.encode(locale.getpreferredencoding()))  # Reference: http://blog.stastnarodina.com/honza-en/spot/python-unicodeencodeerror/
-        success = not process.returncode
+            pollProcess(process, True)
+            stdout, stderr = process.communicate()
+            success = not process.returncode
+        except (IOError, OSError), ex:
+            success = False
+            stderr = getSafeExString(ex)
 
         if success:
-            import lib.core.settings
+            logger.info("%s the latest revision '%s'" % ("already at" if "Already" in stdout else "updated to", getRevisionNumber()))
-            _ = lib.core.settings.REVISION = getRevisionNumber()
-            logger.info("%s the latest revision '%s'" % ("already at" if "Already" in stdout else "updated to", _))
         else:
-            logger.error("update could not be completed ('%s')" % re.sub(r"\W+", " ", stderr).strip())
+            if "Not a git repository" in stderr:
+                errMsg = "not a valid git repository. Please checkout the 'sqlmapproject/sqlmap' repository "
+                errMsg += "from GitHub (e.g. 'git clone --depth 1 https://github.com/sqlmapproject/sqlmap.git sqlmap')"
+                logger.error(errMsg)
+            else:
+                logger.error("update could not be completed ('%s')" % re.sub(r"\W+", " ", stderr).strip())
 
     if not success:
         if IS_WIN:

lib/core/wordlist.py

@@ -1,16 +1,16 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
 import os
 import zipfile
 
+from lib.core.common import getSafeExString
 from lib.core.exception import SqlmapDataException
 from lib.core.exception import SqlmapInstallationException
-from lib.core.settings import UNICODE_ENCODING
 
 class Wordlist(object):
     """
@@ -41,7 +41,13 @@ class Wordlist(object):
         else:
             self.current = self.filenames[self.index]
             if os.path.splitext(self.current)[1].lower() == ".zip":
-                _ = zipfile.ZipFile(self.current, 'r')
+                try:
+                    _ = zipfile.ZipFile(self.current, 'r')
+                except zipfile.error, ex:
+                    errMsg = "something appears to be wrong with "
+                    errMsg += "the file '%s' ('%s'). Please make " % (self.current, getSafeExString(ex))
+                    errMsg += "sure that you haven't made any changes to it"
+                    raise SqlmapInstallationException, errMsg
                 if len(_.namelist()) == 0:
                     errMsg = "no file(s) inside '%s'" % self.current
                     raise SqlmapDataException(errMsg)
@@ -64,17 +70,13 @@ class Wordlist(object):
             try:
                 retVal = self.iter.next().rstrip()
             except zipfile.error, ex:
-                errMsg = "something seems to be wrong with "
+                errMsg = "something appears to be wrong with "
-                errMsg += "the file '%s' ('%s'). Please make " % (self.current, ex)
+                errMsg += "the file '%s' ('%s'). Please make " % (self.current, getSafeExString(ex))
                 errMsg += "sure that you haven't made any changes to it"
                 raise SqlmapInstallationException, errMsg
             except StopIteration:
                 self.adjust()
                 retVal = self.iter.next().rstrip()
-            try:
-                retVal = retVal.decode(UNICODE_ENCODING)
-            except UnicodeDecodeError:
-                continue
             if not self.proc_count or self.counter % self.proc_count == self.proc_id:
                 break
         return retVal

lib/core/xmldump.py

@@ -1,536 +0,0 @@
-#!/usr/bin/env python
-
-import codecs
-import os
-import re
-import xml
-
-import xml.sax.saxutils as saxutils
-
-from lib.core.common import getUnicode
-from lib.core.data import conf
-from lib.core.data import kb
-from lib.core.data import logger
-from lib.core.exception import SqlmapFilePathException
-from lib.core.settings import UNICODE_ENCODING
-from thirdparty.prettyprint import prettyprint
-from xml.dom.minidom import Document
-from xml.parsers.expat import ExpatError
-
-TECHNIC_ELEM_NAME = "Technic"
-TECHNICS_ELEM_NAME = "Technics"
-BANNER_ELEM_NAME = "Banner"
-COLUMNS_ELEM_NAME = "DatabaseColumns"
-COLUMN_ELEM_NAME = "Column"
-CELL_ELEM_NAME = "Cell"
-COLUMN_ATTR = "column"
-ROW_ELEM_NAME = "Row"
-TABLES_ELEM_NAME = "tables"
-DATABASE_COLUMNS_ELEM = "DB"
-DB_TABLES_ELEM_NAME = "DBTables"
-DB_TABLE_ELEM_NAME = "DBTable"
-IS_DBA_ELEM_NAME = "isDBA"
-FILE_CONTENT_ELEM_NAME = "FileContent"
-DB_ATTR = "db"
-UNKNOWN_COLUMN_TYPE = "unknown"
-USER_SETTINGS_ELEM_NAME = "UserSettings"
-USER_SETTING_ELEM_NAME = "UserSetting"
-USERS_ELEM_NAME = "Users"
-USER_ELEM_NAME = "User"
-DB_USER_ELEM_NAME = "DBUser"
-SETTINGS_ELEM_NAME = "Settings"
-DBS_ELEM_NAME = "DBs"
-DB_NAME_ELEM_NAME = "DBName"
-DATABASE_ELEM_NAME = "Database"
-TABLE_ELEM_NAME = "Table"
-DB_TABLE_VALUES_ELEM_NAME = "DBTableValues"
-DB_VALUES_ELEM = "DBValues"
-QUERIES_ELEM_NAME = "Queries"
-QUERY_ELEM_NAME = "Query"
-REGISTERY_ENTRIES_ELEM_NAME = "RegistryEntries"
-REGISTER_DATA_ELEM_NAME = "RegisterData"
-DEFAULT_DB = "All"
-MESSAGE_ELEM = "Message"
-MESSAGES_ELEM_NAME = "Messages"
-ERROR_ELEM_NAME = "Error"
-LST_ELEM_NAME = "List"
-LSTS_ELEM_NAME = "Lists"
-CURRENT_USER_ELEM_NAME = "CurrentUser"
-CURRENT_DB_ELEM_NAME = "CurrentDB"
-MEMBER_ELEM = "Member"
-ADMIN_USER = "Admin"
-REGULAR_USER = "User"
-STATUS_ELEM_NAME = "Status"
-RESULTS_ELEM_NAME = "Results"
-UNHANDLED_PROBLEM_TYPE = "Unhandled"
-NAME_ATTR = "name"
-TYPE_ATTR = "type"
-VALUE_ATTR = "value"
-SUCESS_ATTR = "success"
-NAME_SPACE_ATTR = 'http://www.w3.org/2001/XMLSchema-instance'
-XMLNS_ATTR = "xmlns:xsi"
-SCHEME_NAME = "sqlmap.xsd"
-SCHEME_NAME_ATTR = "xsi:noNamespaceSchemaLocation"
-CHARACTERS_TO_ENCODE = range(32) + range(127, 256)
-ENTITIES = {'"': '"', "'": "'"}
-
-class XMLDump(object):
-    '''
-    This class purpose is to dump the data into an xml Format.
-    The format of the xml file is described in the scheme file xml/sqlmap.xsd
-    '''
-
-    def __init__(self):
-        self._outputFile = None
-        self._outputFP = None
-        self.__root = None
-        self.__doc = Document()
-
-    def _addToRoot(self, element):
-        '''
-        Adds element to the root element
-        '''
-        self.__root.appendChild(element)
-
-    def __write(self, data, n=True):
-        '''
-        Writes the data into the file
-        '''
-        if n:
-            self._outputFP.write("%s\n" % data)
-        else:
-            self._outputFP.write("%s " % data)
-
-        self._outputFP.flush()
-
-        kb.dataOutputFlag = True
-
-    def _getRootChild(self, elemName):
-        '''
-        Returns the child of the root with the described name
-        '''
-        elements = self.__root.getElementsByTagName(elemName)
-        if elements:
-            return elements[0]
-
-        return elements
-
-    def _createTextNode(self, data):
-        '''
-        Creates a text node with utf8 data inside.
-        The text is escaped to an fit the xml text Format.
-        '''
-        if data is None:
-            return self.__doc.createTextNode(u'')
-        else:
-            escaped_data = saxutils.escape(data, ENTITIES)
-            return self.__doc.createTextNode(escaped_data)
-
-    def _createAttribute(self, attrName, attrValue):
-        '''
-        Creates an attribute node with utf8 data inside.
-        The text is escaped to an fit the xml text Format.
-        '''
-        attr = self.__doc.createAttribute(attrName)
-        if attrValue is None:
-            attr.nodeValue = u''
-        else:
-            attr.nodeValue = getUnicode(attrValue)
-        return attr
-
-    def string(self, header, data, sort=True):
-        '''
-        Adds string element to the xml.
-        '''
-        if isinstance(data, (list, tuple, set)):
-            self.lister(header, data, sort)
-            return
-
-        messagesElem = self._getRootChild(MESSAGES_ELEM_NAME)
-        if (not(messagesElem)):
-            messagesElem = self.__doc.createElement(MESSAGES_ELEM_NAME)
-            self._addToRoot(messagesElem)
-
-        if data:
-            data = self._formatString(data)
-        else:
-            data = ""
-
-        elem = self.__doc.createElement(MESSAGE_ELEM)
-        elem.setAttributeNode(self._createAttribute(TYPE_ATTR, header))
-        elem.appendChild(self._createTextNode(data))
-        messagesElem.appendChild(elem)
-
-    def lister(self, header, elements, sort=True):
-        '''
-        Adds information formatted as list element
-        '''
-        lstElem = self.__doc.createElement(LST_ELEM_NAME)
-        lstElem.setAttributeNode(self._createAttribute(TYPE_ATTR, header))
-        if elements:
-            if sort:
-                try:
-                    elements = set(elements)
-                    elements = list(elements)
-                    elements.sort(key=lambda x: x.lower())
-                except:
-                    pass
-
-            for element in elements:
-                memberElem = self.__doc.createElement(MEMBER_ELEM)
-                lstElem.appendChild(memberElem)
-                if isinstance(element, basestring):
-                    memberElem.setAttributeNode(self._createAttribute(TYPE_ATTR, "string"))
-                    memberElem.appendChild(self._createTextNode(element))
-                elif isinstance(element, (list, tuple, set)):
-                    memberElem.setAttributeNode(self._createAttribute(TYPE_ATTR, "list"))
-                    for e in element:
-                        memberElemStr = self.__doc.createElement(MEMBER_ELEM)
-                        memberElemStr.setAttributeNode(self._createAttribute(TYPE_ATTR, "string"))
-                        memberElemStr.appendChild(self._createTextNode(getUnicode(e)))
-                        memberElem.appendChild(memberElemStr)
-        listsElem = self._getRootChild(LSTS_ELEM_NAME)
-        if not(listsElem):
-            listsElem = self.__doc.createElement(LSTS_ELEM_NAME)
-            self._addToRoot(listsElem)
-        listsElem.appendChild(lstElem)
-
-    def technic(self, technicType, data):
-        '''
-        Adds information about the technic used to extract data from the db
-        '''
-        technicElem = self.__doc.createElement(TECHNIC_ELEM_NAME)
-        technicElem.setAttributeNode(self._createAttribute(TYPE_ATTR, technicType))
-        textNode = self._createTextNode(data)
-        technicElem.appendChild(textNode)
-        technicsElem = self._getRootChild(TECHNICS_ELEM_NAME)
-        if not(technicsElem):
-            technicsElem = self.__doc.createElement(TECHNICS_ELEM_NAME)
-            self._addToRoot(technicsElem)
-        technicsElem.appendChild(technicElem)
-
-    def banner(self, data):
-        '''
-        Adds information about the database banner to the xml.
-        The banner contains information about the type and the version of the database.
-        '''
-        bannerElem = self.__doc.createElement(BANNER_ELEM_NAME)
-        bannerElem.appendChild(self._createTextNode(data))
-        self._addToRoot(bannerElem)
-
-    def currentUser(self, data):
-        '''
-        Adds information about the current database user to the xml
-        '''
-        currentUserElem = self.__doc.createElement(CURRENT_USER_ELEM_NAME)
-        textNode = self._createTextNode(data)
-        currentUserElem.appendChild(textNode)
-        self._addToRoot(currentUserElem)
-
-    def currentDb(self, data):
-        '''
-        Adds information about the current database is use to the xml
-        '''
-        currentDBElem = self.__doc.createElement(CURRENT_DB_ELEM_NAME)
-        textNode = self._createTextNode(data)
-        currentDBElem.appendChild(textNode)
-        self._addToRoot(currentDBElem)
-
-    def dba(self, isDBA):
-        '''
-        Adds information to the xml that indicates whether the user has DBA privileges
-        '''
-        isDBAElem = self.__doc.createElement(IS_DBA_ELEM_NAME)
-        isDBAElem.setAttributeNode(self._createAttribute(VALUE_ATTR, getUnicode(isDBA)))
-        self._addToRoot(isDBAElem)
-
-    def users(self, users):
-        '''
-        Adds a list of the existing users to the xml
-        '''
-        usersElem = self.__doc.createElement(USERS_ELEM_NAME)
-        if isinstance(users, basestring):
-            users = [users]
-        if users:
-            for user in users:
-                userElem = self.__doc.createElement(DB_USER_ELEM_NAME)
-                usersElem.appendChild(userElem)
-                userElem.appendChild(self._createTextNode(user))
-        self._addToRoot(usersElem)
-
-    def dbs(self, dbs):
-        '''
-        Adds a list of the existing databases to the xml
-        '''
-        dbsElem = self.__doc.createElement(DBS_ELEM_NAME)
-        if dbs:
-            for db in dbs:
-                dbElem = self.__doc.createElement(DB_NAME_ELEM_NAME)
-                dbsElem.appendChild(dbElem)
-                dbElem.appendChild(self._createTextNode(db))
-        self._addToRoot(dbsElem)
-
-    def userSettings(self, header, userSettings, subHeader):
-        '''
-        Adds information about the user's settings to the xml.
-        The information can be user's passwords, privileges and etc..
-        '''
-        self._areAdmins = set()
-        userSettingsElem = self._getRootChild(USER_SETTINGS_ELEM_NAME)
-        if (not(userSettingsElem)):
-            userSettingsElem = self.__doc.createElement(USER_SETTINGS_ELEM_NAME)
-            self._addToRoot(userSettingsElem)
-
-        userSettingElem = self.__doc.createElement(USER_SETTING_ELEM_NAME)
-        userSettingElem.setAttributeNode(self._createAttribute(TYPE_ATTR, header))
-
-        if isinstance(userSettings, (tuple, list, set)):
-            self._areAdmins = userSettings[1]
-            userSettings = userSettings[0]
-
-        users = userSettings.keys()
-        users.sort(key=lambda x: x.lower())
-
-        for user in users:
-            userElem = self.__doc.createElement(USER_ELEM_NAME)
-            userSettingElem.appendChild(userElem)
-            if user in self._areAdmins:
-                userElem.setAttributeNode(self._createAttribute(TYPE_ATTR, ADMIN_USER))
-            else:
-                userElem.setAttributeNode(self._createAttribute(TYPE_ATTR, REGULAR_USER))
-
-            settings = userSettings[user]
-
-            settings.sort()
-
-            for setting in settings:
-                settingsElem = self.__doc.createElement(SETTINGS_ELEM_NAME)
-                settingsElem.setAttributeNode(self._createAttribute(TYPE_ATTR, subHeader))
-                settingTextNode = self._createTextNode(setting)
-                settingsElem.appendChild(settingTextNode)
-                userElem.appendChild(settingsElem)
-        userSettingsElem.appendChild(userSettingElem)
-
-    def dbTables(self, dbTables):
-        '''
-        Adds information of the existing db tables to the xml
-        '''
-        if not isinstance(dbTables, dict):
-            self.string(TABLES_ELEM_NAME, dbTables)
-            return
-
-        dbTablesElem = self.__doc.createElement(DB_TABLES_ELEM_NAME)
-
-        for db, tables in dbTables.items():
-            tables.sort(key=lambda x: x.lower())
-            dbElem = self.__doc.createElement(DATABASE_ELEM_NAME)
-            dbElem.setAttributeNode(self._createAttribute(NAME_ATTR, db))
-            dbTablesElem.appendChild(dbElem)
-            for table in tables:
-                tableElem = self.__doc.createElement(DB_TABLE_ELEM_NAME)
-                tableElem.appendChild(self._createTextNode(table))
-                dbElem.appendChild(tableElem)
-        self._addToRoot(dbTablesElem)
-
-    def dbTableColumns(self, tableColumns):
-        '''
-        Adds information about the columns of the existing tables to the xml
-        '''
-
-        columnsElem = self._getRootChild(COLUMNS_ELEM_NAME)
-        if not(columnsElem):
-            columnsElem = self.__doc.createElement(COLUMNS_ELEM_NAME)
-
-        for db, tables in tableColumns.items():
-            if not db:
-                db = DEFAULT_DB
-            dbElem = self.__doc.createElement(DATABASE_COLUMNS_ELEM)
-            dbElem.setAttributeNode(self._createAttribute(NAME_ATTR, db))
-            columnsElem.appendChild(dbElem)
-
-            for table, columns in tables.items():
-                tableElem = self.__doc.createElement(TABLE_ELEM_NAME)
-                tableElem.setAttributeNode(self._createAttribute(NAME_ATTR, table))
-
-                colList = columns.keys()
-                colList.sort(key=lambda x: x.lower())
-
-                for column in colList:
-                    colType = columns[column]
-                    colElem = self.__doc.createElement(COLUMN_ELEM_NAME)
-                    if colType is not None:
-                        colElem.setAttributeNode(self._createAttribute(TYPE_ATTR, colType))
-                    else:
-                        colElem.setAttributeNode(self._createAttribute(TYPE_ATTR, UNKNOWN_COLUMN_TYPE))
-                    colElem.appendChild(self._createTextNode(column))
-                    tableElem.appendChild(colElem)
-
-        self._addToRoot(columnsElem)
-
-    def dbTableValues(self, tableValues):
-        '''
-        Adds the values of specific table to the xml.
-        The values are organized according to the relevant row and column.
-        '''
-        tableElem = self.__doc.createElement(DB_TABLE_VALUES_ELEM_NAME)
-        if (tableValues is not None):
-            db = tableValues["__infos__"]["db"]
-            if not db:
-                db = "All"
-            table = tableValues["__infos__"]["table"]
-
-            count = int(tableValues["__infos__"]["count"])
-            columns = tableValues.keys()
-            columns.sort(key=lambda x: x.lower())
-
-            tableElem.setAttributeNode(self._createAttribute(DB_ATTR, db))
-            tableElem.setAttributeNode(self._createAttribute(NAME_ATTR, table))
-
-            for i in range(count):
-                rowElem = self.__doc.createElement(ROW_ELEM_NAME)
-                tableElem.appendChild(rowElem)
-                for column in columns:
-                    if column != "__infos__":
-                        info = tableValues[column]
-                        value = info["values"][i]
-
-                        if re.search("^[\ *]*$", value):
-                            value = "NULL"
-
-                        cellElem = self.__doc.createElement(CELL_ELEM_NAME)
-                        cellElem.setAttributeNode(self._createAttribute(COLUMN_ATTR, column))
-                        cellElem.appendChild(self._createTextNode(value))
-                        rowElem.appendChild(cellElem)
-
-        dbValuesElem = self._getRootChild(DB_VALUES_ELEM)
-        if (not(dbValuesElem)):
-            dbValuesElem = self.__doc.createElement(DB_VALUES_ELEM)
-            self._addToRoot(dbValuesElem)
-
-        dbValuesElem.appendChild(tableElem)
-
-        logger.info("Table '%s.%s' dumped to XML file" % (db, table))
-
-    def dbColumns(self, dbColumns, colConsider, dbs):
-        '''
-        Adds information about the columns
-        '''
-        for column in dbColumns.keys():
-            printDbs = {}
-            for db, tblData in dbs.items():
-                for tbl, colData in tblData.items():
-                    for col, dataType in colData.items():
-                        if column in col:
-                            if db in printDbs:
-                                if tbl in printDbs[db]:
-                                    printDbs[db][tbl][col] = dataType
-                                else:
-                                    printDbs[db][tbl] = {col: dataType}
-                            else:
-                                printDbs[db] = {}
-                                printDbs[db][tbl] = {col: dataType}
-
-                            continue
-
-        self.dbTableColumns(printDbs)
-
-    def query(self, query, queryRes):
-        '''
-        Adds details of an executed query to the xml.
-        The query details are the query itself and its results.
-        '''
-        queryElem = self.__doc.createElement(QUERY_ELEM_NAME)
-        queryElem.setAttributeNode(self._createAttribute(VALUE_ATTR, query))
-        queryElem.appendChild(self._createTextNode(queryRes))
-        queriesElem = self._getRootChild(QUERIES_ELEM_NAME)
-        if (not(queriesElem)):
-            queriesElem = self.__doc.createElement(QUERIES_ELEM_NAME)
-            self._addToRoot(queriesElem)
-        queriesElem.appendChild(queryElem)
-
-    def registerValue(self, registerData):
-        '''
-        Adds information about an extracted registry key to the xml
-        '''
-        registerElem = self.__doc.createElement(REGISTER_DATA_ELEM_NAME)
-        registerElem.appendChild(self._createTextNode(registerData))
-        registriesElem = self._getRootChild(REGISTERY_ENTRIES_ELEM_NAME)
-        if (not(registriesElem)):
-            registriesElem = self.__doc.createElement(REGISTERY_ENTRIES_ELEM_NAME)
-            self._addToRoot(registriesElem)
-        registriesElem.appendChild(registerElem)
-
-    def rFile(self, filePath, data):
-        '''
-        Adds an extracted file's content to the xml
-        '''
-        fileContentElem = self.__doc.createElement(FILE_CONTENT_ELEM_NAME)
-        fileContentElem.setAttributeNode(self._createAttribute(NAME_ATTR, filePath))
-        fileContentElem.appendChild(self._createTextNode(data))
-        self._addToRoot(fileContentElem)
-
-    def setOutputFile(self):
-        '''
-        Initiates the xml file from the configuration.
-        '''
-        if (conf.xmlFile):
-            try:
-                self._outputFile = conf.xmlFile
-                self.__root = None
-
-                if os.path.exists(self._outputFile):
-                    try:
-                        self.__doc = xml.dom.minidom.parse(self._outputFile)
-                        self.__root = self.__doc.childNodes[0]
-                    except ExpatError:
-                        self.__doc = Document()
-
-                self._outputFP = codecs.open(self._outputFile, "w+", UNICODE_ENCODING)
-
-                if self.__root is None:
-                    self.__root = self.__doc.createElementNS(NAME_SPACE_ATTR, RESULTS_ELEM_NAME)
-                    self.__root.setAttributeNode(self._createAttribute(XMLNS_ATTR, NAME_SPACE_ATTR))
-                    self.__root.setAttributeNode(self._createAttribute(SCHEME_NAME_ATTR, SCHEME_NAME))
-                    self.__doc.appendChild(self.__root)
-            except IOError:
-                raise SqlmapFilePathException("Wrong filename provided for saving the xml file: %s" % conf.xmlFile)
-
-    def getOutputFile(self):
-        return self._outputFile
-
-    def finish(self, resultStatus, resultMsg=""):
-        '''
-        Finishes the dumper operation:
-        1. Adds the session status to the xml
-        2. Writes the xml to the file
-        3. Closes the xml file
-        '''
-        if ((self._outputFP is not None) and not(self._outputFP.closed)):
-            statusElem = self.__doc.createElement(STATUS_ELEM_NAME)
-            statusElem.setAttributeNode(self._createAttribute(SUCESS_ATTR, getUnicode(resultStatus)))
-
-            if not resultStatus:
-                errorElem = self.__doc.createElement(ERROR_ELEM_NAME)
-
-                if isinstance(resultMsg, Exception):
-                    errorElem.setAttributeNode(self._createAttribute(TYPE_ATTR, type(resultMsg).__name__))
-                else:
-                    errorElem.setAttributeNode(self._createAttribute(TYPE_ATTR, UNHANDLED_PROBLEM_TYPE))
-
-                errorElem.appendChild(self._createTextNode(getUnicode(resultMsg)))
-                statusElem.appendChild(errorElem)
-
-            self._addToRoot(statusElem)
-            self.__write(prettyprint.formatXML(self.__doc, encoding=UNICODE_ENCODING))
-            self._outputFP.close()
-
-
-def closeDumper(status, msg=""):
-    """
-    Closes the dumper of the session
-    """
-
-    if hasattr(conf, "dumper") and hasattr(conf.dumper, "finish"):
-        conf.dumper.finish(status, msg)
-
-dumper = XMLDump()

lib/parse/__init__.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/parse/banner.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/parse/cmdline.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -17,6 +17,7 @@ from optparse import SUPPRESS_HELP
 
 from lib.core.common import checkDeprecatedOptions
 from lib.core.common import checkSystemEncoding
+from lib.core.common import dataToStdout
 from lib.core.common import expandMnemonics
 from lib.core.common import getUnicode
 from lib.core.data import cmdLineOptions
@@ -30,20 +31,25 @@ from lib.core.settings import BASIC_HELP_ITEMS
 from lib.core.settings import DUMMY_URL
 from lib.core.settings import IS_WIN
 from lib.core.settings import MAX_HELP_OPTION_LENGTH
+from lib.core.settings import UNICODE_ENCODING
 from lib.core.settings import VERSION_STRING
 from lib.core.shell import autoCompletion
 from lib.core.shell import clearHistory
 from lib.core.shell import loadHistory
 from lib.core.shell import saveHistory
 
-def cmdLineParser():
+def cmdLineParser(argv=None):
     """
     This function parses the command line parameters and arguments
     """
 
+    if not argv:
+        argv = sys.argv
+
     checkSystemEncoding()
 
-    _ = getUnicode(os.path.basename(sys.argv[0]), encoding=sys.getfilesystemencoding())
+    # Reference: https://stackoverflow.com/a/4012683 (Note: previously used "...sys.getfilesystemencoding() or UNICODE_ENCODING")
+    _ = getUnicode(os.path.basename(argv[0]), encoding=sys.stdin.encoding)
 
     usage = "%s%s [options]" % ("python " if not IS_WIN else "", \
             "\"%s\"" % _ if " " in _ else _)
@@ -141,12 +147,21 @@ def cmdLineParser():
                            help="HTTP authentication credentials "
                                 "(name:password)")
 
-        request.add_option("--auth-private", dest="authPrivate",
+        request.add_option("--auth-file", dest="authFile",
-                           help="HTTP authentication PEM private key file")
+                           help="HTTP authentication PEM cert/private key file")
 
         request.add_option("--ignore-401", dest="ignore401", action="store_true",
                           help="Ignore HTTP Error 401 (Unauthorized)")
 
+        request.add_option("--ignore-proxy", dest="ignoreProxy", action="store_true",
+                           help="Ignore system default proxy settings")
+
+        request.add_option("--ignore-redirects", dest="ignoreRedirects", action="store_true",
+                          help="Ignore redirection attempts")
+
+        request.add_option("--ignore-timeouts", dest="ignoreTimeouts", action="store_true",
+                          help="Ignore connection timeouts")
+
         request.add_option("--proxy", dest="proxy",
                            help="Use a proxy to connect to the target URL")
 
@@ -157,9 +172,6 @@ def cmdLineParser():
         request.add_option("--proxy-file", dest="proxyFile",
                            help="Load proxy list from a file")
 
-        request.add_option("--ignore-proxy", dest="ignoreProxy", action="store_true",
-                           help="Ignore system default proxy settings")
-
         request.add_option("--tor", dest="tor",
                                   action="store_true",
                                   help="Use Tor anonymity network")
@@ -168,7 +180,7 @@ def cmdLineParser():
                                   help="Set Tor proxy port other than default")
 
         request.add_option("--tor-type", dest="torType",
-                                  help="Set Tor proxy type (HTTP (default), SOCKS4 or SOCKS5)")
+                                  help="Set Tor proxy type (HTTP, SOCKS4 or SOCKS5 (default))")
 
         request.add_option("--check-tor", dest="checkTor",
                                   action="store_true",
@@ -256,7 +268,10 @@ def cmdLineParser():
                              help="Skip testing for given parameter(s)")
 
         injection.add_option("--skip-static", dest="skipStatic", action="store_true",
-                             help="Skip testing parameters that not appear dynamic")
+                             help="Skip testing parameters that not appear to be dynamic")
+
+        injection.add_option("--param-exclude", dest="paramExclude",
+                           help="Regexp to exclude parameters from testing (e.g. \"ses\")")
 
         injection.add_option("--dbms", dest="dbms",
                              help="Force back-end DBMS to this value")
@@ -356,7 +371,7 @@ def cmdLineParser():
         techniques.add_option("--union-from", dest="uFrom",
                               help="Table to use in FROM part of UNION query SQL injection")
 
-        techniques.add_option("--dns-domain", dest="dnsName",
+        techniques.add_option("--dns-domain", dest="dnsDomain",
                               help="Domain name used for DNS exfiltration attack")
 
         techniques.add_option("--second-order", dest="secondOrder",
@@ -461,14 +476,17 @@ def cmdLineParser():
                                help="Exclude DBMS system databases when "
                                     "enumerating tables")
 
+        enumeration.add_option("--pivot-column", dest="pivotColumn",
+                               help="Pivot column name")
+
         enumeration.add_option("--where", dest="dumpWhere",
                                help="Use WHERE condition while table dumping")
 
         enumeration.add_option("--start", dest="limitStart", type="int",
-                               help="First query output entry to retrieve")
+                               help="First dump table entry to retrieve")
 
         enumeration.add_option("--stop", dest="limitStop", type="int",
-                               help="Last query output entry to retrieve")
+                               help="Last dump table entry to retrieve")
 
         enumeration.add_option("--first", dest="firstChar", type="int",
                                help="First query output word character to retrieve")
@@ -600,9 +618,6 @@ def cmdLineParser():
         general = OptionGroup(parser, "General", "These options can be used "
                              "to set some general working parameters")
 
-        #general.add_option("-x", dest="xmlFile",
-        #                    help="Dump the data into an XML file")
-
         general.add_option("-s", dest="sessionFile",
                             help="Load session from a stored (.sqlite) file")
 
@@ -614,9 +629,16 @@ def cmdLineParser():
                             action="store_true",
                             help="Never ask for user input, use the default behaviour")
 
+        general.add_option("--binary-fields", dest="binaryFields",
+                          help="Result fields having binary values (e.g. \"digest\")")
+
         general.add_option("--charset", dest="charset",
                             help="Force character encoding used for data retrieval")
 
+        general.add_option("--check-internet", dest="checkInternet",
+                            action="store_true",
+                            help="Check Internet connection before assessing the target")
+
         general.add_option("--crawl", dest="crawlDepth", type="int",
                             help="Crawl the website starting from the target URL")
 
@@ -632,8 +654,7 @@ def cmdLineParser():
 
         general.add_option("--eta", dest="eta",
                             action="store_true",
-                            help="Display for each output the "
+                            help="Display for each output the estimated time of arrival")
-                                 "estimated time of arrival")
 
         general.add_option("--flush-session", dest="flushSession",
                             action="store_true",
@@ -647,6 +668,9 @@ def cmdLineParser():
                             action="store_true",
                             help="Ignore query results stored in session file")
 
+        general.add_option("--har", dest="harFile",
+                           help="Log all HTTP traffic into a HAR file")
+
         general.add_option("--hex", dest="hexConvert",
                             action="store_true",
                             help="Use DBMS hex function(s) for data retrieval")
@@ -659,11 +683,7 @@ def cmdLineParser():
                                   action="store_true",
                                   help="Parse and display DBMS error messages from responses")
 
-        general.add_option("--pivot-column", dest="pivotColumn",
+        general.add_option("--save", dest="saveConfig",
-                               help="Pivot column name")
-
-        general.add_option("--save", dest="saveCmdline",
-                            action="store_true",
                             help="Save options to a configuration INI file")
 
         general.add_option("--scope", dest="scope",
@@ -672,6 +692,9 @@ def cmdLineParser():
         general.add_option("--test-filter", dest="testFilter",
                            help="Select tests by payloads and/or titles (e.g. ROW)")
 
+        general.add_option("--test-skip", dest="testSkip",
+                           help="Skip tests by payloads and/or titles (e.g. BENCHMARK)")
+
         general.add_option("--update", dest="updateAll",
                             action="store_true",
                             help="Update sqlmap")
@@ -719,20 +742,26 @@ def cmdLineParser():
                                   action="store_true",
                                   help="Work in offline mode (only use session data)")
 
-        miscellaneous.add_option("--page-rank", dest="pageRank",
-                                  action="store_true",
-                                  help="Display page rank (PR) for Google dork results")
-
         miscellaneous.add_option("--purge-output", dest="purgeOutput",
                                   action="store_true",
                                   help="Safely remove all content from output directory")
 
+        miscellaneous.add_option("--skip-waf", dest="skipWaf",
+                                  action="store_true",
+                                  help="Skip heuristic detection of WAF/IPS/IDS protection")
+
         miscellaneous.add_option("--smart", dest="smart",
                                   action="store_true",
                                   help="Conduct thorough tests only if positive heuristic(s)")
 
         miscellaneous.add_option("--sqlmap-shell", dest="sqlmapShell", action="store_true",
-                            help="Prompt for an interactive sqlmap shell")
+                                  help="Prompt for an interactive sqlmap shell")
+
+        miscellaneous.add_option("--tmp-dir", dest="tmpDir",
+                                  help="Local directory for storing temporary files")
+
+        miscellaneous.add_option("--web-root", dest="webRoot",
+                                  help="Web server document root directory (e.g. \"/var/www\")")
 
         miscellaneous.add_option("--wizard", dest="wizard",
                                   action="store_true",
@@ -742,21 +771,24 @@ def cmdLineParser():
         parser.add_option("--dummy", dest="dummy", action="store_true",
                           help=SUPPRESS_HELP)
 
-        parser.add_option("--pickled-options", dest="pickledOptions",
+        parser.add_option("--murphy-rate", dest="murphyRate", type="int",
+                          help=SUPPRESS_HELP)
+
+        parser.add_option("--disable-precon", dest="disablePrecon", action="store_true",
+                          help=SUPPRESS_HELP)
+
+        parser.add_option("--disable-stats", dest="disableStats", action="store_true",
                           help=SUPPRESS_HELP)
 
         parser.add_option("--profile", dest="profile", action="store_true",
                           help=SUPPRESS_HELP)
 
-        parser.add_option("--binary-fields", dest="binaryFields",
-                          help=SUPPRESS_HELP)
-
-        parser.add_option("--cpu-throttle", dest="cpuThrottle", type="int",
-                          help=SUPPRESS_HELP)
-
         parser.add_option("--force-dns", dest="forceDns", action="store_true",
                           help=SUPPRESS_HELP)
 
+        parser.add_option("--force-threads", dest="forceThreads", action="store_true",
+                          help=SUPPRESS_HELP)
+
         parser.add_option("--smoke-test", dest="smokeTest", action="store_true",
                           help=SUPPRESS_HELP)
 
@@ -768,6 +800,14 @@ def cmdLineParser():
 
         parser.add_option("--run-case", dest="runCase", help=SUPPRESS_HELP)
 
+        # API options
+        parser.add_option("--api", dest="api", action="store_true",
+                          help=SUPPRESS_HELP)
+
+        parser.add_option("--taskid", dest="taskid", help=SUPPRESS_HELP)
+
+        parser.add_option("--database", dest="database", help=SUPPRESS_HELP)
+
         parser.add_option_group(target)
         parser.add_option_group(request)
         parser.add_option_group(optimization)
@@ -786,31 +826,33 @@ def cmdLineParser():
 
         # Dirty hack to display longer options without breaking into two lines
         def _(self, *args):
-            _ = parser.formatter._format_option_strings(*args)
+            retVal = parser.formatter._format_option_strings(*args)
-            if len(_) > MAX_HELP_OPTION_LENGTH:
+            if len(retVal) > MAX_HELP_OPTION_LENGTH:
-                _ = ("%%.%ds.." % (MAX_HELP_OPTION_LENGTH - parser.formatter.indent_increment)) % _
+                retVal = ("%%.%ds.." % (MAX_HELP_OPTION_LENGTH - parser.formatter.indent_increment)) % retVal
-            return _
+            return retVal
 
         parser.formatter._format_option_strings = parser.formatter.format_option_strings
         parser.formatter.format_option_strings = type(parser.formatter.format_option_strings)(_, parser, type(parser))
 
-        # Dirty hack for making a short option -hh
+        # Dirty hack for making a short option '-hh'
         option = parser.get_option("--hh")
         option._short_opts = ["-hh"]
         option._long_opts = []
 
-        # Dirty hack for inherent help message of switch -h
+        # Dirty hack for inherent help message of switch '-h'
         option = parser.get_option("-h")
         option.help = option.help.capitalize().replace("this help", "basic help")
 
-        argv = []
+        _ = []
         prompt = False
         advancedHelp = True
         extraHeaders = []
 
-        for arg in sys.argv:
+        # Reference: https://stackoverflow.com/a/4012683 (Note: previously used "...sys.getfilesystemencoding() or UNICODE_ENCODING")
-            argv.append(getUnicode(arg, encoding=sys.getfilesystemencoding()))
+        for arg in argv:
+            _.append(getUnicode(arg, encoding=sys.stdin.encoding))
 
+        argv = _
         checkDeprecatedOptions(argv)
 
         prompt = "--sqlmap-shell" in argv
@@ -845,14 +887,14 @@ def cmdLineParser():
                 if not command:
                     continue
                 elif command.lower() == "clear":
-                    clearHistory()                    
+                    clearHistory()
-                    print "[i] history cleared"
+                    dataToStdout("[i] history cleared\n")
                     saveHistory(AUTOCOMPLETE_TYPE.SQLMAP)
                 elif command.lower() in ("x", "q", "exit", "quit"):
                     raise SqlmapShellQuitException
                 elif command[0] != '-':
-                    print "[!] invalid option(s) provided"
+                    dataToStdout("[!] invalid option(s) provided\n")
-                    print "[i] proper example: '-u http://www.site.com/vuln.php?id=1 --banner'"
+                    dataToStdout("[i] proper example: '-u http://www.site.com/vuln.php?id=1 --banner'\n")
                 else:
                     saveHistory(AUTOCOMPLETE_TYPE.SQLMAP)
                     loadHistory(AUTOCOMPLETE_TYPE.SQLMAP)
@@ -864,10 +906,18 @@ def cmdLineParser():
             except ValueError, ex:
                 raise SqlmapSyntaxException, "something went wrong during command line parsing ('%s')" % ex.message
 
-        # Hide non-basic options in basic help case
         for i in xrange(len(argv)):
             if argv[i] == "-hh":
                 argv[i] = "-h"
+            elif len(argv[i]) > 1 and all(ord(_) in xrange(0x2018, 0x2020) for _ in ((argv[i].split('=', 1)[-1].strip() or ' ')[0], argv[i][-1])):
+                dataToStdout("[!] copy-pasting illegal (non-console) quote characters from Internet is, well, illegal (%s)\n" % argv[i])
+                raise SystemExit
+            elif len(argv[i]) > 1 and u"\uff0c" in argv[i].split('=', 1)[-1]:
+                dataToStdout("[!] copy-pasting illegal (non-console) comma characters from Internet is, well, illegal (%s)\n" % argv[i])
+                raise SystemExit
+            elif re.search(r"\A-\w=.+", argv[i]):
+                dataToStdout("[!] potentially miswritten (illegal '=') short option detected ('%s')\n" % argv[i])
+                raise SystemExit
             elif argv[i] == "-H":
                 if i + 1 < len(argv):
                     extraHeaders.append(argv[i + 1])
@@ -877,7 +927,7 @@ def cmdLineParser():
             elif argv[i] == "--version":
                 print VERSION_STRING.split('/')[-1]
                 raise SystemExit
-            elif argv[i] == "-h":
+            elif argv[i] in ("-h", "--help"):
                 advancedHelp = False
                 for group in parser.option_groups[:]:
                     found = False
@@ -889,14 +939,22 @@ def cmdLineParser():
                     if not found:
                         parser.option_groups.remove(group)
 
+        for verbosity in (_ for _ in argv if re.search(r"\A\-v+\Z", _)):
+            try:
+                if argv.index(verbosity) == len(argv) - 1 or not argv[argv.index(verbosity) + 1].isdigit():
+                    conf.verbose = verbosity.count('v') + 1
+                    del argv[argv.index(verbosity)]
+            except (IndexError, ValueError):
+                pass
+
         try:
             (args, _) = parser.parse_args(argv)
         except UnicodeEncodeError, ex:
-            print "\n[!] %s" % ex.object.encode("unicode-escape")
+            dataToStdout("\n[!] %s\n" % ex.object.encode("unicode-escape"))
             raise SystemExit
         except SystemExit:
             if "-h" in argv and not advancedHelp:
-                print "\n[!] to see full list of options run with '-hh'"
+                dataToStdout("\n[!] to see full list of options run with '-hh'\n")
             raise
 
         if extraHeaders:
@@ -915,9 +973,9 @@ def cmdLineParser():
 
         if not any((args.direct, args.url, args.logFile, args.bulkFile, args.googleDork, args.configFile, \
             args.requestFile, args.updateAll, args.smokeTest, args.liveTest, args.wizard, args.dependencies, \
-            args.purgeOutput, args.pickledOptions, args.sitemapUrl)):
+            args.purgeOutput, args.sitemapUrl)):
             errMsg = "missing a mandatory option (-d, -u, -l, -m, -r, -g, -c, -x, --wizard, --update, --purge-output or --dependencies), "
-            errMsg += "use -h for basic or -hh for advanced help"
+            errMsg += "use -h for basic or -hh for advanced help\n"
             parser.error(errMsg)
 
         return args
@@ -928,7 +986,7 @@ def cmdLineParser():
     except SystemExit:
         # Protection against Windows dummy double clicking
         if IS_WIN:
-            print "\nPress Enter to continue...",
+            dataToStdout("\nPress Enter to continue...")
             raw_input()
         raise
 

lib/parse/configfile.py

@@ -1,30 +1,27 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
-import codecs
-
-from ConfigParser import MissingSectionHeaderError
-from ConfigParser import ParsingError
-
 from lib.core.common import checkFile
+from lib.core.common import getSafeExString
 from lib.core.common import getUnicode
 from lib.core.common import openFile
 from lib.core.common import unArrayizeValue
 from lib.core.common import UnicodeRawConfigParser
+from lib.core.data import cmdLineOptions
 from lib.core.data import conf
 from lib.core.data import logger
+from lib.core.enums import OPTION_TYPE
 from lib.core.exception import SqlmapMissingMandatoryOptionException
 from lib.core.exception import SqlmapSyntaxException
 from lib.core.optiondict import optDict
-from lib.core.settings import UNICODE_ENCODING
 
 config = None
 
-def configFileProxy(section, option, boolean=False, integer=False):
+def configFileProxy(section, option, datatype):
     """
     Parse configuration file and save settings into the configuration
     advanced dictionary.
@@ -34,10 +31,12 @@ def configFileProxy(section, option, boolean=False, integer=False):
 
     if config.has_option(section, option):
         try:
-            if boolean:
+            if datatype == OPTION_TYPE.BOOLEAN:
                 value = config.getboolean(section, option) if config.get(section, option) else False
-            elif integer:
+            elif datatype == OPTION_TYPE.INTEGER:
                 value = config.getint(section, option) if config.get(section, option) else 0
+            elif datatype == OPTION_TYPE.FLOAT:
+                value = config.getfloat(section, option) if config.get(section, option) else 0.0
             else:
                 value = config.get(section, option)
         except ValueError, ex:
@@ -73,23 +72,21 @@ def configFileParser(configFile):
         config = UnicodeRawConfigParser()
         config.readfp(configFP)
     except Exception, ex:
-        errMsg = "you have provided an invalid and/or unreadable configuration file ('%s')" % ex.message
+        errMsg = "you have provided an invalid and/or unreadable configuration file ('%s')" % getSafeExString(ex)
         raise SqlmapSyntaxException(errMsg)
 
     if not config.has_section("Target"):
         errMsg = "missing a mandatory section 'Target' in the configuration file"
         raise SqlmapMissingMandatoryOptionException(errMsg)
 
-    condition = not config.has_option("Target", "direct")
+    mandatory = False
-    condition &= not config.has_option("Target", "url")
-    condition &= not config.has_option("Target", "logFile")
-    condition &= not config.has_option("Target", "bulkFile")
-    condition &= not config.has_option("Target", "googleDork")
-    condition &= not config.has_option("Target", "requestFile")
-    condition &= not config.has_option("Target", "sitemapUrl")
-    condition &= not config.has_option("Target", "wizard")
 
-    if condition:
+    for option in ("direct", "url", "logFile", "bulkFile", "googleDork", "requestFile", "sitemapUrl", "wizard"):
+        if config.has_option("Target", option) and config.get("Target", option) or cmdLineOptions.get(option):
+            mandatory = True
+            break
+
+    if not mandatory:
         errMsg = "missing a mandatory option in the configuration file "
         errMsg += "(direct, url, logFile, bulkFile, googleDork, requestFile, sitemapUrl or wizard)"
         raise SqlmapMissingMandatoryOptionException(errMsg)
@@ -97,8 +94,4 @@ def configFileParser(configFile):
     for family, optionData in optDict.items():
         for option, datatype in optionData.items():
             datatype = unArrayizeValue(datatype)
-
+            configFileProxy(family, option, datatype)
-            boolean = datatype == "boolean"
-            integer = datatype == "integer"
-
-            configFileProxy(family, option, boolean, integer)

lib/parse/handler.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/parse/headers.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 

lib/parse/html.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -24,7 +24,8 @@ class HTMLHandler(ContentHandler):
         ContentHandler.__init__(self)
 
         self._dbms = None
-        self._page = page
+        self._page = (page or "")
+        self._lower_page = self._page.lower()
 
         self.dbms = None
 
@@ -33,11 +34,20 @@ class HTMLHandler(ContentHandler):
         threadData.lastErrorPage = (threadData.lastRequestUID, self._page)
 
     def startElement(self, name, attrs):
+        if self.dbms:
+            return
+
         if name == "dbms":
             self._dbms = attrs.get("value")
 
         elif name == "error":
-            if re.search(attrs.get("regexp"), self._page, re.I):
+            regexp = attrs.get("regexp")
+            if regexp not in kb.cache.regex:
+                keywords = re.findall("\w+", re.sub(r"\\.", " ", regexp))
+                keywords = sorted(keywords, key=len)
+                kb.cache.regex[regexp] = keywords[-1].lower()
+
+            if kb.cache.regex[regexp] in self._lower_page and re.search(regexp, self._page, re.I):
                 self.dbms = self._dbms
                 self._markAsErrorPage()
 
@@ -49,6 +59,13 @@ def htmlParser(page):
 
     xmlfile = paths.ERRORS_XML
     handler = HTMLHandler(page)
+    key = hash(page)
+
+    if key in kb.cache.parsedDbms:
+        retVal = kb.cache.parsedDbms[key]
+        if retVal:
+            handler._markAsErrorPage()
+        return retVal
 
     parseXmlFile(xmlfile, handler)
 
@@ -58,6 +75,8 @@ def htmlParser(page):
     else:
         kb.lastParserStatus = None
 
+    kb.cache.parsedDbms[key] = handler.dbms
+
     # generic SQL warning/error messages
     if re.search(r"SQL (warning|error|syntax)", page, re.I):
         handler._markAsErrorPage()

lib/parse/payloads.py

@@ -1,7 +1,7 @@
 #!/usr/bin/env python
 
 """
-Copyright (c) 2006-2015 sqlmap developers (http://sqlmap.org/)
+Copyright (c) 2006-2017 sqlmap developers (http://sqlmap.org/)
 See the file 'doc/COPYING' for copying permission
 """
 
@@ -9,23 +9,25 @@ import os
 
 from xml.etree import ElementTree as et
 
+from lib.core.common import getSafeExString
 from lib.core.data import conf
 from lib.core.data import paths
 from lib.core.datatype import AttribDict
 from lib.core.exception import SqlmapInstallationException
+from lib.core.settings import PAYLOAD_XML_FILES
 
 def cleanupVals(text, tag):
     if tag in ("clause", "where"):
         text = text.split(',')
 
     if isinstance(text, basestring):
-        text = int(text) if text.isdigit() else str(text)
+        text = int(text) if text.isdigit() else text
 
     elif isinstance(text, list):
         count = 0
 
         for _ in text:
-            text[count] = int(_) if _.isdigit() else str(_)
+            text[count] = int(_) if _.isdigit() else _
             count += 1
 
         if len(text) == 1 and tag not in ("clause", "where"):
@@ -73,8 +75,8 @@ def loadBoundaries():
     try:
         doc = et.parse(paths.BOUNDARIES_XML)
     except Exception, ex:
-        errMsg = "something seems to be wrong with "
+        errMsg = "something appears to be wrong with "
-        errMsg += "the file '%s' ('%s'). Please make " % (paths.BOUNDARIES_XML, ex)
+        errMsg += "the file '%s' ('%s'). Please make " % (paths.BOUNDARIES_XML, getSafeExString(ex))
         errMsg += "sure that you haven't made any changes to it"
         raise SqlmapInstallationException, errMsg
 
@@ -82,17 +84,14 @@ def loadBoundaries():
     parseXmlNode(root)
 
 def loadPayloads():
-    payloadFiles = os.listdir(paths.SQLMAP_XML_PAYLOADS_PATH)
+    for payloadFile in PAYLOAD_XML_FILES:
-    payloadFiles.sort()
-
-    for payloadFile in payloadFiles:
         payloadFilePath = os.path.join(paths.SQLMAP_XML_PAYLOADS_PATH, payloadFile)
 
         try:
             doc = et.parse(payloadFilePath)
         except Exception, ex:
-            errMsg = "something seems to be wrong with "
+            errMsg = "something appears to be wrong with "
-            errMsg += "the file '%s' ('%s'). Please make " % (payloadFilePath, ex)
+            errMsg += "the file '%s' ('%s'). Please make " % (payloadFilePath, getSafeExString(ex))
             errMsg += "sure that you haven't made any changes to it"
             raise SqlmapInstallationException, errMsg