Identified - A problematic instance of our player components was identified as the source of the issues. This instance has been temporarily removed from our load balancers while our team works to restore it to full health.
Players should now load properly from other instances.
More information to follow.
May 20, 16:23 UTC
Players should now load properly from other instances.
More information to follow.
May 20, 16:23 UTC
Investigating - We are investigating an issue where Triton Web Radio players could fail to load. When this happens, users will see "An Internal Error Has Occurred" appear instead of the player. Closing the browser or reloading the page sometimes works to load the player.
This issue is only affecting Triton Web players and does not have any impact whatsoever on the live streams themselves or mobile apps using the Triton SDK.
More information to come.
May 20, 15:53 UTC
This issue is only affecting Triton Web players and does not have any impact whatsoever on the live streams themselves or mobile apps using the Triton SDK.
More information to come.
May 20, 15:53 UTC
Update - We have deployed platform update that allows us to improve reporting capabilities. It is now possible to run reports with a start date that is up to 13 months in the past. However, the report's end date will be capped at a maximum of 90 days from the start date. This means that users wishing to run reports that cover a year can now do so by running 4 separate reports.
Users who are in need of reporting coving more than 13 months in the past are instructed to communicate with our Support Team (supportdesk@tritondigital.com).
More information to follow.
May 19, 19:29 UTC
Users who are in need of reporting coving more than 13 months in the past are instructed to communicate with our Support Team (supportdesk@tritondigital.com).
More information to follow.
May 19, 19:29 UTC
Identified - As part of a previous incident (https://status.tritondigital.com/incidents/8my5chtpbpxw), the team had put in place a mitigation mechanism to restrict Tap Explore queries to be a maximum of 90 days in the past. Since the data display issues were resolved, the incident was closed but the temporary measure of limiting query periods to 90 days is still in effect to ensure product stability.
However, work is still being done to restore capabilities for larger queries.
To make tracking of the temporary limitation easier, we are moving this to its own incident.
At present, we cannot give an ETA of when this limitation will be removed.
More information to follow.
Apr 29, 20:35 UTC
However, work is still being done to restore capabilities for larger queries.
To make tracking of the temporary limitation easier, we are moving this to its own incident.
At present, we cannot give an ETA of when this limitation will be removed.
More information to follow.
Apr 29, 20:35 UTC
Update - Our monitoring has shown that data queries to our various products are running as except for peak usage hours which occur in North American morning (from 13:00 to 16:00 UTC which is around 9:00 to 12:00 EST). Users running large queries during those times might experience issues where they get an error message or "No Data" as a result of the query. We advise these users to wait a few minutes and retry their query.
Our teams have identified several data optimization initiatives which are being added to our various development sprints. The situation should improve as these initiatives are deployed over the coming days.
More information to follow.
May 17, 15:06 UTC
Our teams have identified several data optimization initiatives which are being added to our various development sprints. The situation should improve as these initiatives are deployed over the coming days.
More information to follow.
May 17, 15:06 UTC
Identified - We have identified the cause of the issue and area applying mitigation measures.
The majority of queries should be running a bit smoother now and data should be showing. Large queries might still have issues completing. If your query results in No Data, please wait a minute and re-run your query. The situation should improve in the following hours.
More information to follow.
May 12, 22:06 UTC
The majority of queries should be running a bit smoother now and data should be showing. Large queries might still have issues completing. If your query results in No Data, please wait a minute and re-run your query. The situation should improve in the following hours.
More information to follow.
May 12, 22:06 UTC
Investigating - We have become aware of an issue with some of our data mechanisms which is causing certain user interfaces to be slow or show no data when making queries. Please note that no data loss is being incurred by this issue.
Affected components are as follows:
- Podcast Metrics
- Webacst Metrics
- Streaming Metrics (excluding Realtime stats and audience pages)
- Tap Forecasting
- Tap Explore
More information to come.
May 12, 20:17 UTC
Affected components are as follows:
- Podcast Metrics
- Webacst Metrics
- Streaming Metrics (excluding Realtime stats and audience pages)
- Tap Forecasting
- Tap Explore
More information to come.
May 12, 20:17 UTC
Identified - At 23:00 UTC yesterday, after much effort, the team was able to restore connectivity to our Singapore data centre. However, it was discovered that a network switch has failed.
The result is that our Singapore data centre is now operating at half capacity until this equipment can be replaced. Listeners are able to be served from this location and capacity is being managed through our normal platform redundancy to other data centres.
We are estimating 3 to 5 days for this equipment to be delivered to the location and installed.
More information to follow.
May 17, 14:41 UTC
The result is that our Singapore data centre is now operating at half capacity until this equipment can be replaced. Listeners are able to be served from this location and capacity is being managed through our normal platform redundancy to other data centres.
We are estimating 3 to 5 days for this equipment to be delivered to the location and installed.
More information to follow.
May 17, 14:41 UTC
Investigating - We have become aware of an issue with our Singapore data centre which has become unreachable. Our infrastructure team is working towards restoring connectivity. All listeners has been redirected to other data centres.
More information to follow.
May 16, 19:11 UTC
More information to follow.
May 16, 19:11 UTC
AMERICAS
Degraded Performance
East - Live Streaming Delivery
Operational
West - Live Streaming Delivery
Operational
Podcast Delivery
Operational
Live Streaming Ad Delivery
Operational
On-Demand Ad Delivery
Operational
Podcast Ad Delivery
Operational
Yield-Op Ad Delivery
Operational
Legacy Tap UI
Degraded Performance
TAP UI
Operational
Tap Reporting
Degraded Performance
Tap Reporting - Inventory Availability Forecasting
Degraded Performance
Tap Explore
Degraded Performance
Media Buying API
?
Operational
Order Management System (OMS) Proxy
Operational
Yield-Op UI
Operational
Yield-Op Analytics Dashboard (Rill)
Operational
Yield-Op Scheduled Reports
Operational
3rd Party Trackers & Programmatic Impressions
Operational
Metrics API (Advertising + Measurement)
Operational
Podcast Metrics UI
Degraded Performance
Streaming Metrics UI
Degraded Performance
Webcast Metrics UI
Degraded Performance
Webcast Metrics API
Operational
Data Collection Services (LT / LC)
?
Operational
Omny Studio Management Portal
Operational
Omny Studio Management API
Operational
Omny Campaign Manager
Operational
Omny.fm Publishing
Operational
Single Sign On Services
Operational
Login services
Operational
Custom Reports
Operational
Omny Campaign Manager Forecasting Tool
Operational
Triton Console
Operational
3rd party cookie synchronization
Operational
Omny Studio Webhooks Service
Operational
Data Export Services
Operational
EMEA
Degraded Performance
Live Streaming Delivery
Operational
Podcast Delivery
Operational
Live Streaming Ad Delivery
Operational
On-Demand Ad Delivery
Operational
Podcast Ad Delivery
Operational
Yield-Op Ad Delivery
Operational
Legacy Tap UI
Degraded Performance
Tap Reporting
Degraded Performance
Tap Reporting - Inventory Availability Forecasting
Degraded Performance
Tap Explore
Degraded Performance
Media Buying API
Operational
Order Management System (OMS) Proxy
Operational
Yield-Op UI
Operational
Yield-Op Analytics Dashboard (Rill)
Operational
Yield-Op Scheduled Reports
Operational
3rd Party Trackers & Programmatic Impressions
Operational
Metrics API (Advertising + Measurement)
Operational
Podcast Metrics UI
Operational
Streaming Metrics UI
Degraded Performance
Webcast Metrics UI
Degraded Performance
Webcast Metrics API
Degraded Performance
Data Collection Services (LT / LC)
Operational
Omny Studio Management Portal
Operational
Omny Studio Management API
Operational
Omny Campaign Manager
Operational
Omny.fm Publishing
Operational
Single Sign On Services
Operational
Login services
Operational
Custom Reports
Operational
Omny Campaign Manager Forecasting Tool
Operational
Triton Console
Operational
3rd party cookie synchronisation
Operational
TAP UI
Operational
Omny Studio Webhooks Service
Operational
Data Export Services
Operational
APAC
Partial Outage
South East & Pacific - Live Streaming Delivery
Operational
South - Live Streaming Delivery
Partial Outage
Podcast Delivery
Operational