For a deeper look into our DataScope Select SOAP API, look into:

Overview |  Quickstart |  Documentation |  Downloads |  Tutorials

question

Upvotes
Accepted
28 1 1 7

DSS API ScheduleExtract IntradayPricing

I requested IntradayPricing using ScheduleExtraction because required extra decimal information. small number of symbols.

('JP3193400003', 'Isin'), ('JP3216700009', 'Isin'), ('JP3524600008', 'Isin'), ('JP3783700002', 'Isin'), ('JP3886000003', 'Isin')

small number of Fields (Bid Price, Last Price, Previous Close Price, Previoous Close Date, RIC, ISIN)

The response of FileType keeps 'Partial', never changed to 'Full'. the response was following.. Is this the normal behavior? I requested using ISIN but the response don't include ISIN information..

Bid Price,Last Price,Previous Close Price,Previous Close Date,RIC,ISIN ,,,,7485.NG, ,,3960,10/02/2020,5979.NG,JP3216700009 ,,,,5461.NG, ,,,,2058.FU, ,,,,9664.NG,
dss-rest-apidatascope-selectdsshistorical
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
Accepted
23k 22 9 14

Hello @kazuhisa.matsuda,

I just tested what seems to be your request via Postman tool, as on demand request, is this the correct request? (the result should be the same for on demand and for a scheduled extraction)

{ {protocol}}{ {host}}{ {api}}Extractions/ExtractWithNotes

{
  "ExtractionRequest": {
    "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.IntradayPricingExtractionRequest",
    "ContentFieldNames": [
      "Bid Price", 
      "Last Price", 
      "Previous Close Price", 
      "Previous Close Date", 
      "RIC", 
      "ISIN"
    ],
    "IdentifierList": {
      "@odata.type": "#ThomsonReuters.Dss.Api.Extractions.ExtractionRequests.InstrumentIdentifierList",  
      "InstrumentIdentifiers": [
          {"Identifier": "JP3193400003",  "IdentifierType":"Isin"},
          {"Identifier": "JP3216700009",  "IdentifierType":"Isin"},
          {"Identifier":"JP3524600008",  "IdentifierType":"Isin"}, 
          {"Identifier":"JP3783700002",  "IdentifierType":"Isin"}, 
          {"Identifier":"JP3886000003", "IdentifierType": "Isin"}
      ]
    },
    "Condition": { "ScalableCurrency": true }
  }
}

Get 5 rows back as expected.

For example, my notes look like this:

 
"Notes": [         "Extraction Services Version 14.2.41795 (d0fe3ba7b1b1), Built Aug 18 2020 17:24:47\r\nProcessing started at 10/05/2020 15:37:54.\r\nUser ID: 19945\r\nExtraction ID: 480402147\r\nSchedule: _OnD_0x0746ade74316e69f (ID = 0x0746ade74896e69f)\r\nInput List (5 items): _OnD_0x0746ade74316e69f (ID = 0746ade74756e69f) Created: 10/05/2020 15:37:51 Last Modified: 10/05/2020 15:37:51\r\nSchedule Time: 10/05/2020 15:37:51\r\nReport Template (12 fields): _OnD_0x0746ade74316e69f (ID = 0x0746ade745a6e69f) Created: 10/05/2020 15:37:51 Last Modified: 10/05/2020 15:37:51\r\nNo prices needed currency scaling.\r\nReal-time data was snapped at 10/05/2020 15:37:55, it was scheduled to snap at 10/05/2020 15:37:51.\r\nNo embargo required for this report.\r\nProcessing completed successfully at 10/05/2020 15:37:55, taking 0.661 Secs.\r\nExtraction finished at 10/05/2020 15:37:55 UTC, with servers: x09I16, QSHC20 (0.1 secs), QSSHA1 (0.0 secs)\r\nUsage Summary for User 19945, Client 8116, Template Type Intraday Pricing\r\nBase Usage\r\n        Instrument                          Instrument                   Terms          Price\r\n  Count Type                                Subtype                      Source         Source\r\n------- ----------------------------------- ---------------------------- -------------- ----------------------------------------\r\n      5 Equities                                                         N/A            N/A\r\n-------\r\n      5 Total instruments charged.\r\n      0 Instruments with no reported data.\r\n=======\r\n      5 Instruments in the input list.\r\nNo TRPS complex usage to report -- 5 Instruments in the input list had no reported data.\r\nWriting RIC maintenance report.\r\n",         "Identifier,IdentType,Source,RIC,RecordDate,MaintType,OldValue,NewValue,Factor,FactorType\r\n"     ]

As I do not seem to reproduce any partial data, some info that would help:

Is the result of 5 instruments what you would expect to see?

Do you consistently get the issue that you describe, not all instruments in your result, or you have only seen it once?

What do you see in extraction notes that come back with your result?

Do you only see this in scheduled, are you able to run on-demand request do you see the same partial result?

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
13k 32 12 18

@kazuhisa.matsuda, Please post your request query so that we can help analyze it. Thanks.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Upvotes
28 1 1 7

Thank you very much for your investigation.

Let me break down to a few questions.

1. FileType

According to the sample code of tutorial (dataAvailabilityExtraction.py), it checks FileType then process it when it is 'Full' . My case is getting 'Partial', how should I manage this?

== dataAvailabilityExtraction.py ==

if FileType == 'Full':

# Only fetch contents of file that has FileType = Full

urlFile = urlGetExtractedFile + '(\'' + ExtractedFileId + '\')/$value'

resp = requests.get(urlFile, headers=header)

==

== response from Schedule Extraction ==

{'@odata.context': 'https://hosted.datascopeapi.reuters.com/RestApi/v1/$metadata#ExtractedFiles', 'value': [{'ExtractedFileId': 'VjF8fDcxNTQ0MjE4Mg', 'ReportExtractionId': '480502371', 'ScheduleId': '0x0746afb362a6e6a5', 'FileType': 'RicMaintenanceNote', 'ExtractedFileName': '9011414.dataAvailabilitySchedule.20201006.093439.480502371.x13I10.ric.csv', 'LastWriteTimeUtc': '2020-10-06T00:34:41.231Z', 'ContentsExists': True, 'Size': 90, 'ReceivedDateUtc': '2020-10-06T00:34:41.231Z'}, {'ExtractedFileId': 'VjF8fDcxNTQ0MjE4MQ', 'ReportExtractionId': '480502371', 'ScheduleId': '0x0746afb362a6e6a5', 'FileType': 'Partial', 'ExtractedFileName': '9011414.dataAvailabilitySchedule.20201006.093439.480502371.x13I10.0min.csv', 'LastWriteTimeUtc': '2020-10-06T00:34:41.228Z', 'ContentsExists': True, 'Size': 168, 'ReceivedDateUtc': '2020-10-06T00:34:41.228Z'}, {'ExtractedFileId': 'VjF8fDcxNTQ0MjE4MA', 'ReportExtractionId': '480502371', 'ScheduleId': '0x0746afb362a6e6a5', 'FileType': 'Note', 'ExtractedFileName': '9011414.dataAvailabilitySchedule.20201006.093439.480502371.x13I10.csv.notes.txt', 'LastWriteTimeUtc': '2020-10-06T00:34:41.233Z', 'ContentsExists': True, 'Size': 2033, 'ReceivedDateUtc': '2020-10-06T00:34:41.233Z'}]} PARTIAL: 0 Bid Price,Last Price,Previous Close Price,Previous Close Date,RIC,ISIN ,,,,7485.NG, ,,3960,10/02/2020,5979.NG,JP3216700009 ,,,,5461.NG, ,,,,2058.FU, ,,,,9664.NG,

==

2. ISIN data field.


It seems only published ISIN value when other field(s) were updated. WHY?


3. Previous Close (Price/Date)

7485,NG had trade on 2020-10-06, why I don't get the information?


4. The data was gone when I request the same Items (11:03AM)


1601950981954.png (32.1 KiB)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Hi @kazuhisa.matsuda,

For the FileType issue, please verify if the "Enable Early Partial Delivery of Embargoed Reports" option is unchecked on the DSS GUI User Preference. The option is to receive an early partial report instead of full.

Hi again,

I understand that you refer to the Data Availability Trigger for EOD Pricing Extraction article. The purpose of the article is to retrieve only "today" end of day data. The code provided use data availability trigger in schedule extraction to receive today data only when the data is available. If you want previous day data, you need to change the message used for creating schedule.

Upvotes
28 1 1 7

Could you advice me what should I need to change the code for creating schedule which supports previous day's data?

current code is:

schData = OrderedDict([('ListId',listIdToUse),
                       ('Name',"dataAvailabilitySchedule"),
                       ('ReportTemplateId',reportId),
                       ('Recurrence',OrderedDict([
                           ('@odata.type','#ThomsonReuters.Dss.Api.Extractions.Schedules.SingleRecurrence'),
                           ('IsImmediate',True)])),
                       ('Trigger',OrderedDict([('@odata.type','#ThomsonReuters.Dss.Api.Extractions.Schedules.ImmediateTrigger'),('LimitReportToTodaysData',True)]))])
createDataAvailabilitySchedData = json.dumps(schData, sort_keys=False)
currDT = datetime.datetime.strftime(datetime.datetime.now(), "%Y-%m-%d-%H:%M:%S")

resp = requests.post(urlCreateSchedule, data=createDataAvailabilitySchedData, headers=header)
icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Hi @kazuhisa.matsuda,

Your code looks correct, except the 'LimitReportToTodaysData':True which causes the extraction to be generated for today data only. To get latest updated data, you need to change the parameter to False.

Upvotes
23k 22 9 14

Hello @kazuhisa.matsuda,

In my understanding, you would not make changes in code. Please refer to the article referenced by @veerapath.rungruengrayubkul -> Prices availability in End of Day Pricing report template.

In Schedule, you would select "Last Updated" in stead of "Today Only" and select the time of day when prices have already been made available for the exchange that you require.

icon clock
10 |1500

Up to 2 attachments (including images) can be used with a maximum of 5.0 MiB each and 10.0 MiB total.

Click below to post an Idea Post Idea