MARKET.CSV

MARKET.CSV

Getting Started

MARKET.CSV is a command line utility for loading market data for stocks, ETFs, indexes, and options to CSV on a daily basis.

You may load data from Yahoo Finance, Google Finance, and MSN Money.

The MarketData folder contains a ready-to-use solution for loading data for US markets. Use it to a quick start.

The folder also contains a solution for importing data to a Microsoft SQL Server market data database.

You may automate loading data on schedule and import loaded CSV data to databases.

 

You may try all the features during a 15-day trial period.

You may start learning on the following topics:

Change History

Version 2.7, December 13, 2017

Bug fixes:

  • Fixed issues with loading data from Google Finance.

Version 2.6, November 3, 2017

Bug fixes:

  • Registration wizard WebException in the offline mode

Version 2.5, August 22, 2017

Breaking changes:

  • Changed .NET platform from .NET Framework 4.0 Client Profile to .NET Framework 4.5.2.
  • Changes in requirements for Windows 7, Windows Server 2008, and Windows Server 2008 R2.
  • Removed support of Windows XP.

Bug Fixes:

  • Issues with registering products.

Version 2.4, August 1, 2017

Improvements:

  • Improved JSON parser performance

Version 2.3, July 5, 2017

New features:

  • Register mode

Improvements:

  • Updated registration wizard

Version 2.2, May 31, 2017

Update Notes

Please import all the downloaded data before update.

The default configuration is compatible with the previous versions.
However, the newest configuration may break your changes.

The newest version has disabled providers, updated providers, and a lot of new providers.

Please read about new features and test them before use in the production.

New Providers

  • GoogleHistoricalPricesDaily
  • GoogleHistoricalPricesWeekly
  • GoogleIntradayPrices15m
  • GoogleIntradayPrices1m
  • GoogleIntradayPrices30m
  • GoogleIntradayPrices5m
  • GoogleIntradayPrices60m
  • MsnMoneyHistoricalPricesDaily
  • MsnMoneyHistoricalPricesMonthly
  • MsnMoneyHistoricalPricesWeekly
  • MsnMoneyIntradayPrices15m
  • MsnMoneyIntradayPrices1m
  • MsnMoneyIntradayPrices5m
  • MsnMoneyOptions
  • MsnMoneyQuotes
  • YahooAssetProfiles
  • YahooHistoricalPricesDaily
  • YahooHistoricalPricesMonthly
  • YahooHistoricalPricesWeekly
  • YahooIntradayPrices15m
  • YahooIntradayPrices1m
  • YahooIntradayPrices5m
  • YahooIntradayPrices60m
  • YahooQuoteSummary
  • YahooQuoteSummaryETF

Disabled Providers (by Yahoo Finance)

  • YahooDividends
  • YahooHistoricalPrices

Updated Providers

  • YahooIndustries (migrate to YahooAssetProfiles)
  • YahooKeyStatistics (migrate to YahooQuoteSummary)
  • YahooOptions
  • YahooQuotes

New Features

  • New provider configuration parameters for configuring URLs and CSV output fields: <Provider>_URL1, <Provider>_URL2, and <Provider>_Fields
  • New command line modes to print fields and values from JSON files: PrintFields and PrintValues
  • New command line mode to list supported exchanges at Yahoo Finance, Google Finance, and MSN Money: Exchanges
  • New command line mode to list providers: Exchanges
  • New SQL codes in the MarketData\bin folder to create Microsoft SQL Server tables for new providers.

Improvements

  • Ignoring task file tickers that cannot be downloaded.
    For example, YahooOptions ignores currencies and non-US tickers.
    So, you may place all required tickers in one file.
  • Ignored configuration parameters: YahooFinanceHistoricalPricesStartDate and YahooFinanceQuotesFields
    The first belongs to the removed YahooHistoricalPrices provider.
    The second is replaced with a newer configuration.

Bug Fixes

  • Exception messages on parsing HTML responses when JSON is expected.

Version 2.1, May 11, 2017

Improvements

  • Spanish translation in the RegisterProduct wizard
  • Improved documentation

Version 2.0, May 1, 2017

MARKET.CSV 2.0 is a major upgrade of Market Data Downloader.

The new name reflects the main application role, saving data from Yahoo Finance to CSV.

You may upgrade Market Data Downloader to MARKET.CSV for free.

The installer version has been removed as the portable version is the best choice.

Licensing Changes

MARKET.CSV has a free edition that allows getting data for 10 tickers only, without delays. See Edition Comparison.

Registration is required to start a trial.

MARKET.CSV is available as a subscription.

Upgrades to the next major versions will be paid. You may purchase Upgrade Warranty with new purchases to get major upgrades for free.

Improvements

Bug Fixes

  • Fixed NaN values
  • Fixed comma separator in the Market capitalization field

Edition Comparison

MARKET.CSV allows loading market data for stocks, ETFs, indexes, and options into CSV files.

FeatureFreePersonalEnterprise
Data Providers
Yahoo Finance
Google Finance
MSN Money
Market Data
Quotes
Historical and intraday prices
Financial data
Option chains (US markets only)
Features
Number of tickers to download10UnlimitedUnlimited
Licensing
One-time price per copy $99$299
Commercial usex

You may try all the features during a 15-day trial period.

Install and Uninstall

Installing

Copy the MarketData folder from the download package to the C: drive.

Then try quick steps.

MARKET.CSV.exe requires .NET Framework 4 Client Profile or Microsoft .NET Framework 4 installed.

You may download required files if market.csv.exe does not work:

Microsoft .NET Framework 4 Client Profile (Standalone Installer)
http://www.microsoft.com/en-us/download/details.aspx?id=24872

Microsoft .NET Framework 4 Client Profile (Web Installer)
http://www.microsoft.com/en-us/download/details.aspx?id=17113

Updating

To update MARKET.CSV, just replace application files from the Update folder of the downloaded package.

Uninstalling

To uninstall MARKET.CSV, just remove the MarketData folder.

System Requirements

Supported Architectures:

  • x86, x64

Supported Operating Systems:

  • Windows Vista SP2, 7 SP1, 8, 8.1, 10
  • Windows Server 2008 SP2, 2008 R2 SP1, 2012, 2012 R2, 2016

.NET Framework 4.5.2 or higher must be installed.

End-User License Agreement

This End-User License Agreement (EULA) is a legal agreement between you (either an individual or a single entity) and Sergey Vaselenko for any MARKET.CSV software, use examples and documentation (Software) that accompany this EULA.

YOU AGREE TO BE BOUND BY THE TERMS OF THIS EULA BY INSTALLING, COPYING, OR OTHERWISE USING THE SOFTWARE.

IF YOU DO NOT AGREE, DO NOT INSTALL, COPY, OR USE THE SOFTWARE.

Your licensing of Software is in accordance with the terms of the EULA in effect at the time of such licensing. By licensing Software, you accept and agree to the EULA in effect at such time.

  1. GRANT OF LICENSE: Sergey Vaselenko grants you the following rights provided that you comply with all terms and conditions of this EULA:
    1. Trial Period: You may install, use, access, display and run the Software for purposes of evaluation of the Software for a period of fifteen (15) days from the date of installation of the Software to determine if the Software meets your needs. Upon completion of your evaluation or at the end of the 15-day trial period, whichever occurs first, you must do one of four things: either
      1) register the MARKET.CSV Free edition in order to continue use of Software;
      2) purchase license for the MARKET.CSV Personal or Enterprise edition in order to continue use of Software;
      3) purchase a subscription for the MARKET.CSV Personal or Enterprise edition in order to continue use of Software;
      or 4) discontinue use of the Software, and uninstall the Software.
    2. Installation and Use: You may install, use, access, display and run one (1) copy of the Software per license, on a non-exclusive basis and without right of sublicense or concurrent use, on a single computer or machine, such as a workstation.
      The commercial use of the MARKET.CSV Personal edition is not permitted, including the use by professional traders and institutional investors.
    3. Software Transfer: You may transfer the Software to a different internal workstation or user so long as you have purchased a License for each such workstation or user. You may not, however, transfer the Software to a Third Party.
    4. Use of Examples: You may install, access, modify and use Software examples, including Microsoft Excel workbooks, databases, SQL codes, and batch files, for your private or company-internal purposes.
  2. LIMITATIONS: You may not use, copy, modify, display, rent, lease, loan, transfer, distribute, download, merge, or make any translation or derivative work of the Software, except as expressly provided herein. You may not reverse engineer, decompile, or disassemble the Software, except and only to the extent that such activity is expressly permitted by applicable law notwithstanding this limitation.
  3. INTELLECTUAL PROPERTY RIGHTS AND CONFIDENTIALITY: The Software, including methods, processes and/or techniques utilized therein, is owned by, proprietary to and valuable trade secrets of Sergey Vaselenko and is protected by Russian Federation copyright law and international treaties. You agree to take no actions that impair or infringe Sergey Vaselenko's intellectual property rights in the Software. You agree not to remove, efface or obscure any copyright notices, other proprietary markings or confidentiality legends placed upon or contained within the Software.
  4. DISCLAIMER OF WARRANTIES: Sergey Vaselenko disclaims all warranties concerning the Software and Services (if any), express, implied, or statutory, including without limitation, any warranties, duties or conditions of merchantability or fitness for a particular purpose, warranties of reliability or availability, of accuracy or completeness of responses, of results, of workmanlike effort, of lack of viruses, and of lack of negligence, all with regard to the Software, and the provision of or failure to provide support or other services, information, software, and related content through the Software or otherwise arising out of the use of the Software. Sergey Vaselenko does not warrant that the Software will operate in combination with other software products selected by you, or that the Software will operate uninterrupted or error-free. Additionally, Sergey Vaselenko and its suppliers provide the Software and Services AS IS AND WITH ALL FAULTS. THERE IS NO WARRANTY OR CONDITION OF TITLE, QUIET ENJOYMENT, QUIET POSSESSION, CORRESPONDENCE TO DESCRIPTION, OR NONINFRINGEMENT WITH REGARD TO THE SOFTWARE.
  5. NO LIABILITY: TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL SERGEY VASELENKO OR ITS SUPPLIERS BE LIABLE FOR ANY SPECIAL, INCIDENTAL, PUNITIVE, INDIRECT, OR CONSEQUENTIAL DAMAGES WHATSOEVER (INCLUDING, BUT NOT LIMITED TO, DAMAGES FOR LOSS OF PROFITS OR CONFIDENTIAL OR OTHER INFORMATION, FOR LOSS OF DATA, FOR BUSINESS INTERRUPTION, FOR PERSONAL INJURY, FOR LOSS OF PRIVACY, FOR FAILURE TO MEET ANY DUTY INCLUDING OF GOOD FAITH OR OF REASONABLE CARE, FOR NEGLIGENCE, AND FOR ANY OTHER PECUNIARY OR OTHER LOSS WHATSOEVER) ARISING OUT OF OR IN ANY WAY RELATED TO THE USE OF OR INABILITY TO USE THE SOFTWARE, THE PROVISION OF OR FAILURE TO PROVIDE SUPPORT OR OTHER SERVICES, INFORMATION, SOFTWARE, AND RELATED CONTENT THROUGH THE SOFTWARE OR OTHERWISE ARISING OUT OF THE USE OF THE SOFTWARE, OR OTHERWISE UNDER OR IN CONNECTION WITH ANY PROVISION OF THIS EULA, EVEN IN THE EVENT OF THE FAULT, TORT (INCLUDING NEGLIGENCE), MISREPRESENTATION, STRICT LIABILITY, BREACH OF CONTRACT OF SERGEY VASELENKO OR ANY SUPPLIER, AND EVEN IF SERGEY VASELENKO OR ANY SUPPLIER HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
  6. LIMITATION ON REMEDIES: Within the first thirty (30) days after your receipt of the Software, should you encounter and report to Sergey Vaselenko within such time period a reproducible error that causes the Software not to perform in all material respects as set forth in the Software documentation, then Sergey Vaselenko will, at its sole discretion, either: a) resolve the error or malfunction, and modify or replace the Software (if deemed necessary by Sergey Vaselenko); or b) allow you to terminate this EULA with respect to the non-conforming Software and, upon your return of the Software to Sergey Vaselenko, Sergey Vaselenko shall provide you with the lesser of a) the actual damages incurred by you; or b) the amount you paid for the non-conforming Software. The remedies described in this section shall be your sole and exclusive remedies under this EULA.
  7. GENERAL PROVISIONS
    1. Reservation of Rights and Ownership: Sergey Vaselenko reserves all rights not expressly granted to you in this EULA. The Software is licensed, not sold.
    2. Consent to Use of Data: You agree that Sergey Vaselenko and its affiliates may collect and use technical information gathered as part of the Software support services provided to you, if any, related to the Software. Sergey Vaselenko may use this information solely to improve Sergey Vaselenko products or to provide customized services or technologies to you and will not disclose this information in a form that personally identifies you.
    3. Links to Third Party Sites: We may link to third-party sites through the use of the Software. The third-party sites are not under the control of Sergey Vaselenko, and Sergey Vaselenko is not responsible for the contents of any third-party sites, any links contained in third-party sites, or any changes or updates to third-party sites. Sergey Vaselenko is not responsible for webcasting or any other form of transmission received from any third-party sites. Sergey Vaselenko is providing these links to third-party sites to you only as a convenience, and the inclusion of any link does not imply an endorsement by Sergey Vaselenko of the third-party site.
    4. Additional Software/Services: This EULA applies to updates, supplements, add-on components, or Internet-based services components, of the Software that Sergey Vaselenko may provide to you or make available to you after the date you obtain your initial copy of the Software, unless Sergey Vaselenko provides other terms along with the update, supplement, add-on component, or Internet-based services component. Sergey Vaselenko reserves the right to discontinue any Internet-based services provided to you or made available to you through the use of the Software.
    5. Upgrades: To use Software identified as an upgrade, you must first be licensed for the software identified by Sergey Vaselenko as eligible for the upgrade. After upgrading, you may no longer use the software that formed the basis for your upgrade eligibility.
    6. Applicable Law: This EULA is governed by the laws of the Russian Federation. Any legal action or proceeding relating to this EULA shall be instituted in a court of arbitration in the Moscow City, Russian Federation. Sergey Vaselenko and you agree to submit to the jurisdiction of, and agree that venue is proper in, these courts in any such action or proceeding. The prevailing party in any action to enforce this EULA will be entitled to recover its attorney fees and costs in connection with such action.
    7. Waiver: The failure of either party to enforce any of the terms of this EULA shall not be construed as a waiver of future enforcement of that or any other term.
    8. Entire Agreement and Severability: This EULA (including any addendum or amendment to this EULA which is included with the Software) is the entire agreement between you and Sergey Vaselenko relating to the Software and the support services (if any) and it supersedes all prior or contemporaneous oral or written communications, proposals and representations with respect to the Software or any other subject matter covered by this EULA. To the extent the terms of any Sergey Vaselenko policies or programs for support services conflict with the terms of this EULA, the terms of this EULA shall control. If any provision of this EULA is held to be void, invalid, unenforceable or illegal, the other provisions shall continue in full force and effect.
    9. Termination: Without prejudice to any other rights, Sergey Vaselenko may terminate this EULA if you fail to comply with the terms and conditions of this EULA. In such event, you must destroy all copies of the Software and all of its component parts and you will not be entitled to any refund of monies.

Trial Version Limitations

You can use MARKET.CSV without any limit during a 15-day trial period.

You have to activate the trial.

When the trial period has expired, you have the following options:

  1. To register the MARKET.CSV Free edition for free.
  2. To purchase a license and register the MARKET.CSV Personal or Enterprise edition.
  3. To purchase a subscription and register the MARKET.CSV Personal or Enterprise edition.
  4. To stop the use and remove MARKET.CSV from a computer.

See also

Using MARKET.CSV

MARKET.CSV is a command line utility that has two main modes: download and reparse.

In the download mode, the application downloads data, converts data to CSV and makes zip archives of all the data.

In the reparse mode, the application parses archived data and saves the data to CSV.
This mode is mainly used to reparse data with a new version of the utility if the previous version does not work properly.

You may create batch files to run the utility with required parameters and configure automatic downloading on schedule.

The MarketData folder contains a ready-to-use solution for loading data for US markets and importing data into a Microsoft SQL Server database.

Downloading Market Data

In the download mode the utility has the following parameters:

market.csv.exe download <provider> [<symbol file name>] [/config=<configuration file>]

Here is an example of commands for loading data from US exchanges:

market.csv.exe download YahooQuotes                 symbols\us-quotes.txt       /config=configs\us.config
market.csv.exe download MsnMoneyQuotes              symbols\us-quotes.txt       /config=configs\us.config
market.csv.exe download YahooQuoteSummary           symbols\us-quotes.txt       /config=configs\us.config
market.csv.exe download YahooAssetProfiles          symbols\us-profiles.txt     /config=configs\us.config
market.csv.exe download YahooOptions                symbols\us-options.txt      /config=configs\us.config
market.csv.exe download MsnMoneyOptions             symbols\us-options.txt      /config=configs\us.config
market.csv.exe download YahooHistoricalPricesDaily  symbols\us-prices.txt       /config=configs\us.config
market.csv.exe download GoogleHistoricalPricesDaily symbols\us-prices.txt       /config=configs\us.config

You may see that commands use different providers, different symbol list files located at the symbols subfolder, and the same application configuration file located at the configs subfolder.

You may get the list of actual data providers using the command:

market.csv.exe providers

You may place your tickers to symbol files using a text editor like notepad.exe, or export tickers from your database.

The application configuration file contains important settings like market open and close times, market time zones, used folders, etc.

You may omit the configuration file. The default file is market.csv.exe.config.

The commands shown above produce the following result files:

csv\YahooQuotes.csv
csv\MsnMoneyQuotes.csv
csv\YahooQuoteSummary.csv
csv\YahooAssetProfiles.csv
csv\YahooOptions.csv
csv\MsnMoneyOptions.csv
csv\YahooHistoricalPricesDaily.csv
csv\GoogleHistoricalPricesDaily.csv

You may import CSV data to a database or to Excel spreadsheets

See related topics:

Reparsing Downloaded Data

In the reparse mode the utility has the following parameters:

market.csv.exe reparse  <provider> [<date as yyyymmdd>] [/config=<configuration file>]

Here is an example of commands for reparsing downloaded data from US exchanges for all trading dates:

market.csv.exe reparse YahooQuotes                  /config=configs\us.config
market.csv.exe reparse MsnMoneyQuotes               /config=configs\us.config
market.csv.exe reparse YahooQuoteSummary            /config=configs\us.config
market.csv.exe reparse YahooAssetProfiles           /config=configs\us.config
market.csv.exe reparse YahooOptions                 /config=configs\us.config
market.csv.exe reparse MsnMoneyOptions              /config=configs\us.config
market.csv.exe reparse YahooHistoricalPricesDaily   /config=configs\us.config
market.csv.exe reparse GoogleHistoricalPricesDaily  /config=configs\us.config

You may omit the configuration file. The default file is market.csv.exe.config.

See related topics:

Command Line

Command Line Modes:

market.csv.exe download <provider> [<symbol file name>] [/config=<configuration file>]

market.csv.exe reparse  <provider> [<date as yyyymmdd>] [/config=<configuration file>]

market.csv.exe printFields <JSON file>
market.csv.exe printValues <JSON file>

market.csv.exe providers
market.csv.exe exchanges

market.csv.exe register

Examples:

market.csv.exe download YahooFinanceOptions symbols.txt

market.csv.exe reparse  YahooFinanceOptions 20170424

market.csv.exe download YahooFinanceQuotes  symbols\uk-symbols.txt /config=configs\uk.config

market.csv.exe reparse  YahooFinanceQuotes  /config=configs\uk.config

Download Mode

In this mode, the application loads and parses data using the specified provider for the specified symbol list.

The symbol file may be omitted if it is specified in the configuration file.

Reparse Mode

In this mode, the application reparses loaded data (in the downloads folder) for the specified trade date.

If the trade date is not specified, the application reparsed all the files.

This mode is used to reparse data using a new version of MARKET.CSV if the previous version does not parse the data correctly.
In the last case, you may continue to download data and reparse the loaded data with a new version later.

PrintFields Mode

Use this mode to list fields available in the input JSON file.

PrintValues Mode

Use this mode to list fields available in the input JSON file with the values.

Providers Mode

Use this mode to list available providers.

You may use different symbol lists for different providers, different time periods, and different markets.

You may load all the data in one cycle using a batch file. See examples in the MarketData folder.

Exchanges Mode

Use this mode to list exchanges supported by Yahoo Finance, Google Finance, and MSN Money.

Register Mode

Use this mode to launch the Registration Wizard.

/Config Option

This option allows specifying alternative configuration files. This is useful when you load data from several world markets.

Exit Codes

You may use the following exit codes in batch files:

Exit CodeDescription
0Success
1Incomplete command line parameters
2Wrong command line parameters
3Exceptions occurred
4A user pressed Q to stop downloading

Application Configuration File

The application reads settings from the application configuration file.

The default file is market.csv.exe.config. Please always keep it with the executable file, market.csv.exe.

You may specify alternative files using the config option. For example:

market.csv.exe download YahooFinanceQuotes symbols\uk-symbols.txt /config=configs\uk.config

This feature is used for loading different settings for different markets.

The application configuration files may contain the following settings:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    <add key="DefaultSymbolFileName" value="" />
    <add key="MarketOpen" value="09:30:00 -05:00" />
    <add key="MarketClose" value="16:00:00 -05:00" />
    <add key="MarketTimeZone" value="-05:00" />
    <add key="NextRequestDelay" value="500" />
    <add key="RetryCycleDelay" value="5000" />
    <add key="RetryCycleCount" value="3" />
    <add key="Timeout" value="30000" />
    <add key="Accept" value="" />
    <add key="UserAgent" value="" />
    <add key="CsvFolder" value="csv" />
    <add key="DownloadFolder" value="downloads" />
    <add key="ZipFolder" value="zip" />
    <add key="CacheFolder" value="cache" />
    <add key="LogFolder" value="logs" />
    <add key="InputCodePage" value="1252" />
    <add key="OutputCodePage" value="1252" />
    <add key="CsvSeparator" value=";" />
    <add key="CsvDateFormat" value="yyyy-MM-dd" />
    <add key="CsvTimeFormat" value="HH:mm:ss" />
    <add key="<Provider>_URL1" value="<URL>" />
    <add key="<Provider>_URL2" value="<URL>" />
    <add key="<Provider>_Fields" value="[<Field>[ AS <Name>][,...]]" />
  </appSettings>
</configuration>

Application Settings

DefaultSymbolFileName
This string value defines the default symbol file name. If specified, you may omit the file name in the download mode.
MarketOpen
This time value defines the market opening time.
The value is used to calculate trade dates. It is very important.
MarketClose
This time value defines the market closing time.
The value is used to calculate trade dates. It is very important.
MARKET.CSV loads data after closing till opening once.
MarketTimeZone
This string value defines the market time zone.
The time zone is used to calculate the market time of the downloaded data. It is very important.
NextRequestDelay
This integer value defines a delay between data requests in milliseconds.
The minimal value is 500 milliseconds that is used by default.
RetryCycleDelay
This integer value defines a delay between cycles of reloading data with errors in milliseconds.
This time should be enough to reconnect your computer to the Internet.
RetryCycleCount
This integer value defines a number of cycles of loading data.
On the first cycle, the application loads data from the current symbol list.
If an error occurred, the symbol is added to a retry list for the next cycle.
The application tries to reload data the specified number of cycles.
Timeout
This integer value defines the web request timeout in milliseconds.
Accept
This string value defines the web request Accept header.
UserAgent
This string value defines the web request UserAgent header.
CsvFolder
This string value defines the folder for placing parsed CSV data.
The default value is "csv".
DownloadDataFolder
This string value defines the folder for placing downloaded data (in HTML, JSON, or CSV format).
The default value is "downloads".
ZipFolder
This string value defines the folder for placing archives of parsed data.
The default value is "zip".
CacheFolder
This string value defines the folder for placing cached data.
The default value is "cache".
LogFolder
This string value defines the folder for placing log files.
The default value is "log".
InputCodePage
This integer value defines a code page for symbol list files.
The default value is the default Windows code page.
OutputCodePage
This integer value defines a code page for output CSV files.
The default value is the default Windows code page.
CsvSeparator
This string value defines CSV separator.
The default value is a semicolon.
CsvDateFormat
This string value defines the date format for output CSV files as specified at
https://msdn.microsoft.com/en-us/library/8kb3ddd4.aspx.
The default value is "yyyy-MM-dd".
CsvTimeFormat
This string value defines the time format for output CSV files as specified at
https://msdn.microsoft.com/en-us/library/8kb3ddd4.aspx.
The default value is "HH:mm:ss".
YahooFinanceHistoricalPricesStartDate
This date value defines the start date for the YahooFinanceHistoricalPrices and YahooFinanceDividends providers.
Specify the date in the yyyy-MM-dd format. The default value is 2013-12-31.
The parameter is removed since version 2.2.
YahooFinanceQuotesFields
This string value defines fields loaded by the YahooFinanceQuotes provider.
The parameter is removed since version 2.2.
<Provider>_URL1
This string value redefines the default URL with a single parameter (Symbol) used to download data.
Use this option to reflect changes made by data providers like Yahoo Finance, Google Finance, or MSN Money.
Usually, we change such URLs in newer versions to have the download URLs actual.
The URL must contain the '{0}' parameter used to insert the symbol.
For example:
<add key="YahooOptions_URL1" value="http://query1.finance.yahoo.com/v7/finance/options/{0}" />
<Provider>_URL2
This string value redefines the default URL with two parameters (Symbol and ExpDate) used to download data.
Use this option to reflect changes made by data providers like Yahoo Finance, Google Finance, or MSN Money.
The URL must contain the '{0}' and '{1}' parameter used to insert the symbol and the second parameter like an expiration date.
You may customize the parameters as described here.
Use the '{1:x}' form for Unix timestamps.
For example:
<add key="YahooOptions_URL2" value="http://query1.finance.yahoo.com/v7/finance/options/{0}?date={1:x}" />
<Provider>_Fields
This string value allows customizing output CSV parameters including changing names and the column orders.
For example:
<add key="YahooOptions_Fields" value="LoadDate,LoadTime,MarketDate,MarketTime,Symbol,Expiration AS ExpDate,Type,Strike,
    ContractSymbol AS ContractName,lastPrice AS Last,Bid,Ask,Change,PercentChange AS %Change,Volume,
    OpenInterest AS Open Interest,ImpliedVolatility AS Implied Volatility,
    -LastTradeDate,-Currency,-ContractSize,-InTheMoney" />
As you see, you may use multiple lines, the AS keyword to define the output name, and the hyphen at the field start to disable the field.
All the providers have built-in fields like LoadDate or Symbol, and specific fields from a data source like lastPrice or ImpliedVolatility.

MarketData Folder

The MarketData folder contains a ready-to-use solution for

  • Loading key statistics, quotes, industries, historical prices, dividends, and options.
  • Importing data into a Microsoft SQL Server database.

The solution is configured for US exchanges.

You may adapt it to load data from exchanges in other countries like UK, Canada, Europe, or Australia.

Quick Start

  1. Copy the MarketData folder from the downloaded package to the root of drive C:
  2. Run C:\MarketData\RegisterProduct.exe and activate the 15-day trial.
  3. Place your tickers to C:\MarketData\symbols.txt.
  4. Run the C:\MarketData\0-run-all.cmd batch file.
  5. Open CSV files in the C:\MarketData\csv folder, for example, in Microsoft Excel.

You may edit symbols in symbols.txt and run the batch file again.
The application will load data for new symbols and reload data for others if the market is open.

Next Quick Steps

You may test the following steps if you have Microsoft SQL Server installed.

  1. Create database tables using SQL codes from the files:
    C:\MarketData\bin\mssql-create-database-google.sql
    C:\MarketData\bin\mssql-create-database-msn.sql
    C:\MarketData\bin\mssql-create-database-yahoo.sql
  2. Edit the connection credentials and uncomment commands in the C:\MarketData\3-import-data.cmd file.
  3. Run the C:\MarketData\0-run-all.cmd batch file.
  4. Select data from database tables like yahoo.QuoteSummary or yahoo.AssetProfiles.

That's all. You may add your tickers to symbol files and add batch files to Windows Scheduler to load data automatically.

Do not hesitate to contact us, if you need a solution for your database platform.

Subfolders

FolderDescription
csvThe folder contains output CSV files to load into a database.
The files are deleted after a successful import.
cacheThe folder contains outputs of the YahooFinanceHistoricalPrices provider (not actual since version 2.2).
downloadsThe folder contains archives of downloaded data.
Archives contain data for a single provider and trade date.
Keep these files to have the possibility to reparse the loaded data.
zipThe folder contains archives of output CSV data.
You may use these files to import data into a database once again.
If you successfully import data and make a backup of your database, you may delete these files.
logsThe folder contains log files.
You may delete these files at any time.
binThe folder contains SQL codes and format files used to import CSV files into SQL Server.
Also, the folder contains SQL codes to create and remove MarketData Database.

Project Files

FileDescription
symbols.txtThe file contains symbols for download.
You may place your tickers into the file using any text editor, or export symbols from your database.
You may create multiple files for different modes.
For example, you may load quotes for 2000 weekly monitored stocks
and daily option data for 200 most actively traded stocks.
0-run-all.cmdThe batch file executes download and import operations.
Use this file in Windows Schedule for automatic data download.
1-export-symbols.cmdThis is a dummy file where you may add commands to export tickers from your database to symbol files.
2-download-data.cmdThe batch file executes download operations.
The file contains commands for loading key statistics, quotes, industries, historical prices, dividends, and options.
3-import-data.cmdThis file executes import operations.
It contains calls of the sqlcmd utility to execute SQL scripts from the bin folder.
Edit the sqlcmd command if you have another SQL Server instance or other database credentials.
market.csv.exeThe application executable file.
market.csv.exe.configThe default application configuration file.
This file is used if the /config option is not specified.
You may change settings to your primary market.
RegisterProduct.exeThe product registration wizard.
bin\clean-data.cmdThis file cleans the loaded data and related files.
Use it if you want to reset the loaded data completely.
bin\YahooQuotes.fmt
bin\YahooOptions.fmt, etc.
The format files are used by SQL Server to import CSV files.
You may edit these files if you need to change column names.
Reflect such changes to SQL scripts in the bin folder.
bin\mssql-import-YahooQuotes.sql
bin\mssql-import-YahooOptions.sql, etc.
SQL scripts used to import CSV files into SQL Server.
The files contain direct paths to CSV and format files like
C:\MarketData\csv\YahooQuotes.csv
C:\MarketData\bin\YahooQuotes.fmt
Edit such paths if you place the MarketData folder in another location.
bin\mssql-create-database-google.sql
bin\mssql-create-database-msn.sql
bin\mssql-create-database-yahoo.sql
bin\mssql-remove-database.sql
SQL scripts to create and remove MarketData Database.

MarketData Database

MARKET.CSV loads data to CSV.

The MarketData folder contains everything you need to create an SQL Server database and to load data from CSV to the database.

You may copy SQL codes to create the required tables from the files:

  • bin\mssql-create-database-google.sql
  • bin\mssql-create-database-msn.sql
  • bin\mssql-create-database-yahoo.sql

You may use mssql-remove-database.sql to drop tables.

You may edit table structures and default schemas.

Just reflect your changes in the mssql-import-*.sql files used to import data.

Configuring Markets

MARKET.CSV allows loading data for any exchange supported by Yahoo Finance, Google Finance, or MSN Money.

Different markets have different time zones and trading hours that requires configuring.

MARKET.CSV by default is configured for US stock exchanges.

If you load data for exchanges in one country only, you may edit default settings.

If you load data for exchanges in several countries, you have two options:

  1. Create folder copies for different exchanges.
  2. Create configuration files for different exchanges.

The first way is simple. Just copy the folder and edit configuration files and import SQL codes.

The second way seems better as you have all the data in the same folder. It is described below.

Copy the application configuration file, market.csv.exe.config, to a file for your market and change settings.

Change, at least, the following settings:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    ...
    <add key="MarketOpen" value="09:30:00 -05:00" />
    <add key="MarketClose" value="16:00:00 -05:00" />
    <add key="MarketTimeZone" value="-05:00" />
    <add key="DownloadFolder" value="downloads" />
    <add key="ZipFolder" value="zip" />
    ...
  </appSettings>
</configuration>

For example, you may set the following settings for London Stock Exchange (uk.config):

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <appSettings>
    ...
    <add key="MarketOpen" value="08:00:00 +00:00" />
    <add key="MarketClose" value="16:30:00 +00:00" />
    <add key="MarketTimeZone" value="+00:00" />
    <add key="DownloadFolder" value="downloads\uk" />
    <add key="ZipFolder" value="zip\uk" />
    ...
  </appSettings>
</configuration>

After that, you may run market.csv.exe with the created configuration file like this:

market.csv.exe download YahooFinanceQuotes symbols\uk-symbols.txt /config=configs\uk.config

market.csv.exe reparse  YahooFinanceQuotes /config=configs\uk.config

Automating Data Loading

You may automatically load data using Windows Schedule.

First of all, create a batch file that includes all the required commands.
You may find ready-to-use batch files in the MarketData folder.

Then use the following commands to create a scheduled task.

1. Open Control Panel, Administrative Tools, Task Scheduler.

2. Select Task Scheduler Library, right click and open New Folder... Create a new folder, for example, MarketData.

3. Select the created folder, right click and open Create a Basic Task.

4. Type a task name, for example, Load US Market Data. Click Next.

Type Load US Market Data and click Next.

5. Select Weekly and click Next.

Select Weekly and click Next.

6. Specify the start time and required days. Click Next.

Specify time and days. Click Next.

7. Select Start a program. Click Next.

Select Start a program. Click Next.

8. Click Browse... and select the batch file to run, copy the batch folder and paste it to the Start in field. Click Next.

Click Browse... and open the target batch file. Fill Start in. Click Next.

9. Click Finish.

Check data and click Finish.

That's all. You may click on the created task, and check or edit task parameters.

Importing CSV to Databases

Every database platform has specific tools for importing CSV files into databases.

So, you may use such tools if you know them.

The MarketData folder contains a ready-to-use solution for Microsoft SQL Server.

Do not hesitate to contact us if you need a solution for your database platform.

Importing CSV to Excel

You may import downloaded CSV data into Microsoft Excel spreadsheets using native Excel features:

  • Just open a CSV file,
  • or use Data, Get External Data, From Text to import the data.

In the second case, you may refresh the data.

Frequently Asked Questions

What is the purpose of the files in "downloads" folder?
The files can be used for reparsing data if the source page format has been changed.
Usually, you just download a new version of MARKET.CSV and reparse the data.
Can I delete archive files in the downloads folder?
Yes. However, be sure that all the files are parsed to CSV correctly.
Do you release new versions to parse changed web pages correctly?
We make everything to release new versions in this case as soon as possible.
Do you guarantee that the product will work in the future?
No. Data providers may change data formats or prevent using products like MARKET.CSV.
We hope that you will use our product for a long time, and we do everything to address new challenges.
Do I pay monthly or yearly fee?
No.
Where you get the data?
MARKET.CSV loads data from Yahoo Finance web pages and web services directly.
We do not provide any data.
Is it legal to load data using your software?
Please read Terms of Service at the corresponding website and answer this question yourself.
Our software communicates with the websites and web services with required delays.
Do not launch several copies of the product at the same time. Otherwise, you may overload data provider services, and your IP will be banned.
Can you add a data provider for a web page on my request?
Please contact us. Technically, the product can load and parse any web page.

Product Registration

MARKET.CSV requires registration to start a trial and to use it after the trial period.

To register, run RegisterProduct.exe in the MarketData folder.

Selecting Edition

You may register the trial version for free; the product code is not required.

To register the MARKET.CSV Personal or Enterprise edition fill in the product code received after purchasing.

MARKET.CSV Registration - Select the edition

Licensee Data

Please fill in the registration form like this:

MARKET.CSV Registration - Fill personal data

The Next button is enabled when all the required fields are filled.

Don't forget to check the required field about the personal data use.

Online Registration

This step allows you to check your email address. If the email is valid, click Next.

You may return to the previous step using the Back button.

MARKET.CSV Registration - Check the email

After clicking the Next button, the MARKET.CSV connects to the registration server.

If the connection is successful, the final step screen is shown.

If any error occurred during connection, you might try to register the product later or try to register the product by email.

MARKET.CSV Registration - The final step

You may set the registration for all users of the computer. This action requires administrator privileges.

Click Finish.

Registration by Email

If you choose the Register by email option on the Licensee Data step, the following screen is displayed:

MARKET.CSV Registration - Check and start the registration email

The Next button starts the default email program, creates a registration email, and activates the next step.
Don't forget to send the email.

If starting of the email program has failed, you may create the registration email manually using the shown registration data.
Please carefully copy the To, Subject, and the Message fields.

The registration server sends the reply in a couple of seconds; but you may close the dialog box and open it again, in the same step.

Please copy the activation code from the received registration email and paste it into the Activation code field.

MARKET.CSV Registration - Paste the activation code

The Next button is enabled when the activation code is valid.

Click Next to continue.

The wizard checks the registration data and confirms the registration.

MARKET.CSV Registration - The final step

You may set the registration for all users of the computer. This action requires administrator privileges.

Click Finish.