Live Status\Incident History

December 5, 2024

Experiencing video processing getting stuck randomly

Outage

Opened: Dec 5, 2024, 5:17 AM UTC

Duration: 4h 35m 58s


  • Opened
    Dec 5, 2024, 5:17 AM UTC

    We are currently investigating this issue.

  • Resolved
    Dec 5, 2024, 9:52 AM UTC

    There was an issue with our AWS Lambda crashing because of memory limits. We have increased the memory limit and have manually processed the affected videos.

  • Resolved
    Dec 13, 2024, 2:54 PM UTC

    What happened: - A 2-hour recording was uploaded, which had a size of 1.6 GB. - Our AWS Lambda was configured to have 3GB RAM and 10GB of ephemeral storage. - Because of the size and processing of the recording, AWS Lambda ran out of memory and crashed. - The crash caused a failure in generating a transcript for this particular recording. So why other recordings were not processed: - Contrary to our understanding, AWS lambda can use the same execution environment for subsequent invocations. - For processing the WEBM file stored on S3, we download it and store it in the /tmp directory of the Lambda environment. Temporary files generated because of the subsequent operations to this file are also stored in the same /tmp directory. We manually clear the /tmp directory after every execution. But the /tmp directory did not get cleared because of the crash. Because of that, a few of the subsequent invocations also crashed because storage crossed the 10GB limit. Action items related to the above issue: - We manually processed the files for which the transcoding had failed - https://github.com/bigbinary/neeto-record-web/blob/main/docs/ops/transcoding-files-manually.md - We have bumped up the RAM to 7 GB. Ephemeral storage is already set to a maximum of 10GB. Other action items identified: - https://github.com/bigbinary/neeto-record-web/issues/2658 - Transcript generation was taking a lot of time for longer videos. We identified the fix and made transcript generation faster. - We bumped up the AWS Lambda timeout from 5 to 15 mins. This will ensure that larger files won't timeout. - https://github.com/bigbinary/neeto-record-web/issues/2657 - We created a cron job that runs every hour to process unprocessed files. - https://github.com/bigbinary/neeto-record-web/issues/2655 - We updated the code in AWS Lambda to clear the /tmp directory at the start of the execution as well. - https://github.com/bigbinary/neeto-record-web/issues/2659 - We set up an alarm to report crashes in AWS Lambda.

December 4, 2024

Healthcheck Assertion Failed

Outage

Opened: Dec 4, 2024, 1:04 PM UTC

Duration: 6m 45s


  • Opened
    Dec 4, 2024, 1:04 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Dec 4, 2024, 1:11 PM UTC

    Issue resolved on monitor recovery.

November 27, 2024

Site Unreachable

Outage

Opened: Nov 27, 2024, 6:38 PM UTC

Duration: 4m 33s


  • Opened
    Nov 27, 2024, 6:38 PM UTC

    Issue opened on monitor failure.

  • Unresolved
    Nov 27, 2024, 6:40 PM UTC

    We are investigating this issue.

  • Unresolved
    Nov 27, 2024, 6:42 PM UTC

    It is a "rate limiting issue". Disabled rack attack for now. Will investigate root cause later.

  • Resolved
    Nov 27, 2024, 6:43 PM UTC

    Issue resolved on monitor recovery.

November 22, 2024

Healthcheck Assertion Failed

Outage

Opened: Nov 22, 2024, 7:52 PM UTC

Duration: 1m 59s


  • Opened
    Nov 22, 2024, 7:52 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Nov 22, 2024, 7:54 PM UTC

    Issue resolved on monitor recovery.

November 15, 2024

Healthcheck Assertion Failed

Outage

Opened: Nov 15, 2024, 8:17 AM UTC

Duration: 8m 6s


  • Opened
    Nov 15, 2024, 8:17 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Nov 15, 2024, 8:25 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Nov 15, 2024, 7:35 AM UTC

Duration: 1m 49s


  • Opened
    Nov 15, 2024, 7:35 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Nov 15, 2024, 7:37 AM UTC

    Issue resolved on monitor recovery.

November 13, 2024

Site Unreachable

Outage

Opened: Nov 13, 2024, 8:43 AM UTC

Duration: 1m 56s


  • Opened
    Nov 13, 2024, 8:43 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Nov 13, 2024, 8:45 AM UTC

    Issue resolved on monitor recovery.

October 24, 2024

Healthcheck Assertion Failed

Outage

Opened: Oct 24, 2024, 6:47 PM UTC

Duration: 46m 1s


  • Opened
    Oct 24, 2024, 6:47 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Oct 24, 2024, 7:33 PM UTC

    Issue resolved on monitor recovery.

October 22, 2024

Site Unreachable

Outage

Opened: Oct 22, 2024, 12:27 PM UTC

Duration: 17h 13m 41s


  • Opened
    Oct 22, 2024, 12:27 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Oct 23, 2024, 5:40 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Oct 22, 2024, 10:36 AM UTC

Duration: 11m 19s


  • Opened
    Oct 22, 2024, 10:36 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Oct 22, 2024, 10:47 AM UTC

    Issue resolved on monitor recovery.

October 18, 2024

Site Unreachable

Outage

Opened: Oct 18, 2024, 9:15 AM UTC

Duration: 4m 59s


  • Opened
    Oct 18, 2024, 9:15 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Oct 18, 2024, 9:20 AM UTC

    Issue resolved on monitor recovery.

Site Unreachable

Outage

Opened: Oct 18, 2024, 9:14 AM UTC

Duration: 6m 51s


  • Opened
    Oct 18, 2024, 9:14 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Oct 18, 2024, 9:21 AM UTC

    Issue resolved on monitor recovery.

Site Unreachable

Outage

Opened: Oct 18, 2024, 9:10 AM UTC

Duration: 19m 8s


  • Opened
    Oct 18, 2024, 9:10 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Oct 18, 2024, 9:19 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Oct 18, 2024, 10:06 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Oct 18, 2024, 10:17 AM UTC

    Issue resolved on monitor recovery.

Site Unreachable

Outage

Opened: Oct 18, 2024, 9:00 AM UTC

Duration: 10m 4s


  • Opened
    Oct 18, 2024, 9:00 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Oct 18, 2024, 9:10 AM UTC

    Issue resolved on monitor recovery.

Site Unreachable

Outage

Opened: Oct 18, 2024, 8:55 AM UTC

Duration: 12m 59s


  • Opened
    Oct 18, 2024, 8:55 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Oct 18, 2024, 9:08 AM UTC

    Issue resolved on monitor recovery.

September 30, 2024

Site Unreachable

Outage

Opened: Sep 30, 2024, 9:25 AM UTC

Duration: 5m 2s


  • Opened
    Sep 30, 2024, 9:25 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 30, 2024, 9:30 AM UTC

    Issue resolved on monitor recovery.

September 24, 2024

Healthcheck Assertion Failed

Outage

Opened: Sep 24, 2024, 5:29 PM UTC

Duration: 1m 20s


  • Opened
    Sep 24, 2024, 5:29 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 24, 2024, 5:30 PM UTC

    Issue resolved on monitor recovery.

Site Unreachable

Outage

Opened: Sep 24, 2024, 5:15 PM UTC

Duration: 157h 8m 50s


  • Opened
    Sep 24, 2024, 5:15 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Oct 1, 2024, 6:24 AM UTC

    Issue resolved on monitor recovery.

Site Unreachable

Outage

Opened: Sep 24, 2024, 5:05 PM UTC

Duration: 10m 37s


  • Opened
    Sep 24, 2024, 5:05 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 24, 2024, 5:16 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Sep 24, 2024, 5:05 PM UTC

Duration: 6m 16s


  • Opened
    Sep 24, 2024, 5:05 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 24, 2024, 5:11 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Sep 24, 2024, 5:04 PM UTC

Duration: 1h 8m 51s


  • Opened
    Sep 24, 2024, 5:04 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 24, 2024, 5:09 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Sep 24, 2024, 5:32 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Sep 24, 2024, 5:36 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Sep 24, 2024, 5:53 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Sep 24, 2024, 5:57 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Sep 24, 2024, 6:08 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Sep 24, 2024, 6:08 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Sep 24, 2024, 6:24 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Sep 24, 2024, 6:25 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Sep 24, 2024, 6:31 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Sep 24, 2024, 6:35 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Sep 24, 2024, 6:41 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Sep 24, 2024, 6:44 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Sep 24, 2024, 7:17 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Sep 24, 2024, 8:05 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Sep 24, 2024, 9:54 AM UTC

Duration: 1h 47s


  • Opened
    Sep 24, 2024, 9:54 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 24, 2024, 10:54 AM UTC

    Issue resolved on monitor recovery.

September 12, 2024

Healthcheck Assertion Failed

Outage

Opened: Sep 12, 2024, 4:29 PM UTC

Duration: 1m 55s


  • Opened
    Sep 12, 2024, 4:29 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 12, 2024, 4:31 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Sep 12, 2024, 4:09 PM UTC

Duration: 5m 25s


  • Opened
    Sep 12, 2024, 4:09 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 12, 2024, 4:15 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Sep 12, 2024, 9:29 AM UTC

Duration: 8m 18s


  • Opened
    Sep 12, 2024, 9:29 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 12, 2024, 9:37 AM UTC

    Issue resolved on monitor recovery.

September 11, 2024

Healthcheck Assertion Failed

Outage

Opened: Sep 11, 2024, 4:28 PM UTC

Duration: 5m 32s


  • Opened
    Sep 11, 2024, 4:28 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 11, 2024, 4:33 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Sep 11, 2024, 4:45 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Sep 11, 2024, 4:46 PM UTC

    Issue resolved on monitor recovery.

September 10, 2024

Healthcheck Assertion Failed

Outage

Opened: Sep 10, 2024, 8:25 AM UTC

Duration: 38m 32s


  • Opened
    Sep 10, 2024, 8:25 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 10, 2024, 9:03 AM UTC

    Issue resolved on monitor recovery.

September 3, 2024

Healthcheck Assertion Failed

Outage

Opened: Sep 3, 2024, 3:43 PM UTC

Duration: 29m 25s


  • Opened
    Sep 3, 2024, 3:43 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 3, 2024, 4:13 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Sep 3, 2024, 7:13 AM UTC

Duration: 7m 52s


  • Opened
    Sep 3, 2024, 7:13 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 3, 2024, 7:21 AM UTC

    Issue resolved on monitor recovery.

September 2, 2024

Healthcheck Assertion Failed

Outage

Opened: Sep 2, 2024, 6:10 PM UTC

Duration: 2m 5s


  • Opened
    Sep 2, 2024, 6:10 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Sep 2, 2024, 6:12 PM UTC

    Issue resolved on monitor recovery.

August 24, 2024

Healthcheck Assertion Failed

Outage

Opened: Aug 24, 2024, 11:40 AM UTC

Duration: 2m 27s


  • Opened
    Aug 24, 2024, 11:40 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Aug 24, 2024, 11:42 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Aug 24, 2024, 8:51 AM UTC

Duration: 25m 1s


  • Opened
    Aug 24, 2024, 8:51 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Aug 24, 2024, 9:16 AM UTC

    Issue resolved on monitor recovery.

August 18, 2024

Healthcheck Assertion Failed

Outage

Opened: Aug 18, 2024, 6:19 PM UTC

Duration: 28m 58s


  • Opened
    Aug 18, 2024, 6:19 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Aug 18, 2024, 6:48 PM UTC

    Issue resolved on monitor recovery.

August 16, 2024

Site Unreachable

Outage

Opened: Aug 16, 2024, 1:33 PM UTC

Duration: 2m 57s


  • Opened
    Aug 16, 2024, 1:33 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Aug 16, 2024, 1:36 PM UTC

    Issue resolved on monitor recovery.

August 14, 2024

Healthcheck Assertion Failed

Outage

Opened: Aug 14, 2024, 3:09 PM UTC

Duration: 45m 56s


  • Opened
    Aug 14, 2024, 3:09 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Aug 14, 2024, 3:55 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Aug 14, 2024, 3:07 PM UTC

Duration: 1h 14m 18s


  • Opened
    Aug 14, 2024, 3:07 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Aug 14, 2024, 4:21 PM UTC

    Issue resolved on monitor recovery.

August 8, 2024

Healthcheck Assertion Failed

Outage

Opened: Aug 8, 2024, 11:11 AM UTC

Duration: 10m 41s


  • Opened
    Aug 8, 2024, 11:11 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Aug 8, 2024, 11:21 AM UTC

    Issue resolved on monitor recovery.

July 25, 2024

Healthcheck Assertion Failed

Outage

Opened: Jul 25, 2024, 5:45 AM UTC

Duration: 12m 16s


  • Opened
    Jul 25, 2024, 5:45 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 25, 2024, 5:52 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 25, 2024, 6:09 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 25, 2024, 6:11 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 25, 2024, 6:25 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 25, 2024, 6:27 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 25, 2024, 6:42 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 25, 2024, 6:44 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 25, 2024, 4:59 AM UTC

Duration: 22m 58s


  • Opened
    Jul 25, 2024, 4:59 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 25, 2024, 5:02 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 25, 2024, 5:12 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 25, 2024, 5:15 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 25, 2024, 5:45 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 25, 2024, 5:47 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 25, 2024, 6:02 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 25, 2024, 6:06 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 25, 2024, 6:21 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 25, 2024, 6:25 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 25, 2024, 6:43 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 25, 2024, 6:44 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 25, 2024, 7:00 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 25, 2024, 7:02 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 25, 2024, 7:23 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 25, 2024, 7:28 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 25, 2024, 4:34 AM UTC

Duration: 4m 36s


  • Opened
    Jul 25, 2024, 4:34 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 25, 2024, 4:39 AM UTC

    Issue resolved on monitor recovery.

July 22, 2024

Healthcheck Assertion Failed

Outage

Opened: Jul 22, 2024, 1:41 PM UTC

Duration: 57m 21s


  • Opened
    Jul 22, 2024, 1:41 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 22, 2024, 2:38 PM UTC

    Issue resolved on monitor recovery.

July 16, 2024

Site Unreachable

Missing Data

Opened: Jul 16, 2024, 1:33 AM UTC

Duration: 12h 43m 58s


  • Opened
    Jul 16, 2024, 1:33 AM UTC

    Issue opened on monitor failure.

  • Investigating
    Jul 16, 2024, 9:28 AM UTC
  • Resolved
    Jul 16, 2024, 2:17 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 16, 2024, 1:30 AM UTC

Duration: 7h 28m 52s


  • Opened
    Jul 16, 2024, 1:30 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 16, 2024, 8:59 AM UTC

    Issue resolved on monitor recovery.

Site Unreachable

Outage

Opened: Jul 16, 2024, 1:22 AM UTC

Duration: 12h 45m 1s


  • Opened
    Jul 16, 2024, 1:22 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 16, 2024, 2:07 PM UTC

    Issue resolved on monitor recovery.

July 11, 2024

Healthcheck Assertion Failed

Outage

Opened: Jul 11, 2024, 8:57 PM UTC

Duration: 2m 59s


  • Opened
    Jul 11, 2024, 8:57 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 11, 2024, 9:00 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 11, 2024, 8:56 PM UTC

Duration: 2m 19s


  • Opened
    Jul 11, 2024, 8:56 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 11, 2024, 8:59 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 11, 2024, 8:56 PM UTC

Duration: 4m 50s


  • Opened
    Jul 11, 2024, 8:56 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 11, 2024, 9:01 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 11, 2024, 8:56 PM UTC

Duration: 4m 9s


  • Opened
    Jul 11, 2024, 8:56 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 11, 2024, 9:00 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 11, 2024, 8:56 PM UTC

Duration: 4m 30s


  • Opened
    Jul 11, 2024, 8:56 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 11, 2024, 9:01 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 11, 2024, 8:56 PM UTC

Duration: 4m 23s


  • Opened
    Jul 11, 2024, 8:56 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 11, 2024, 9:00 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 11, 2024, 8:55 PM UTC

Duration: 5m 14s


  • Opened
    Jul 11, 2024, 8:55 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 11, 2024, 9:00 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 11, 2024, 8:55 PM UTC

Duration: 5m 25s


  • Opened
    Jul 11, 2024, 8:55 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 11, 2024, 9:00 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 11, 2024, 3:10 PM UTC

Duration: 1m 44s


  • Opened
    Jul 11, 2024, 3:10 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 11, 2024, 3:12 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 11, 2024, 3:10 PM UTC

Duration: 2m 33s


  • Opened
    Jul 11, 2024, 3:10 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 11, 2024, 3:12 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 11, 2024, 3:26 AM UTC

Duration: 2m 51s


  • Opened
    Jul 11, 2024, 3:26 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 11, 2024, 3:29 AM UTC

    Issue resolved on monitor recovery.

July 2, 2024

Healthcheck Assertion Failed

Outage

Opened: Jul 2, 2024, 7:33 PM UTC

Duration: 9m 46s


  • Opened
    Jul 2, 2024, 7:33 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 2, 2024, 7:43 PM UTC

    Issue resolved on monitor recovery.

Site Unreachable

Outage

Opened: Jul 2, 2024, 3:14 AM UTC

Duration: 4m 23s


  • Opened
    Jul 2, 2024, 3:14 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 2, 2024, 3:18 AM UTC

    Issue resolved on monitor recovery.

July 1, 2024

Healthcheck Assertion Failed

Outage

Opened: Jul 1, 2024, 3:42 AM UTC

Duration: 32m 39s


  • Opened
    Jul 1, 2024, 3:42 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 1, 2024, 4:15 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 1, 2024, 3:05 AM UTC

Duration: 5m 53s


  • Opened
    Jul 1, 2024, 3:05 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 1, 2024, 3:09 AM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jul 1, 2024, 4:00 AM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jul 1, 2024, 4:02 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jul 1, 2024, 2:29 AM UTC

Duration: 1m 13s


  • Opened
    Jul 1, 2024, 2:29 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jul 1, 2024, 2:30 AM UTC

    Issue resolved on monitor recovery.

June 30, 2024

Healthcheck Assertion Failed

Outage

Opened: Jun 30, 2024, 11:28 PM UTC

Duration: 4m 7s


  • Opened
    Jun 30, 2024, 11:28 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 30, 2024, 11:32 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 30, 2024, 12:36 PM UTC

Duration: 2m 25s


  • Opened
    Jun 30, 2024, 12:36 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 30, 2024, 12:38 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 30, 2024, 12:28 PM UTC

Duration: 5m 17s


  • Opened
    Jun 30, 2024, 12:28 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 30, 2024, 12:30 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jun 30, 2024, 12:33 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jun 30, 2024, 12:36 PM UTC

    Issue resolved on monitor recovery.

June 28, 2024

Healthcheck Assertion Failed

Outage

Opened: Jun 28, 2024, 4:22 PM UTC

Duration: 1m 42s


  • Opened
    Jun 28, 2024, 4:22 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 28, 2024, 4:24 PM UTC

    Issue resolved on monitor recovery.

June 21, 2024

Healthcheck Assertion Failed

Outage

Opened: Jun 21, 2024, 3:53 AM UTC

Duration: 1m 41s


  • Opened
    Jun 21, 2024, 3:53 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 21, 2024, 3:55 AM UTC

    Issue resolved on monitor recovery.

June 14, 2024

Healthcheck Assertion Failed

Outage

Opened: Jun 14, 2024, 9:17 PM UTC

Duration: 3m 58s


  • Opened
    Jun 14, 2024, 9:17 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 14, 2024, 9:21 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 14, 2024, 10:08 AM UTC

Duration: 14m 44s


  • Opened
    Jun 14, 2024, 10:08 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 14, 2024, 10:22 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 14, 2024, 10:07 AM UTC

Duration: 14m 33s


  • Opened
    Jun 14, 2024, 10:07 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 14, 2024, 10:22 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 14, 2024, 10:07 AM UTC

Duration: 14m 42s


  • Opened
    Jun 14, 2024, 10:07 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 14, 2024, 10:22 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 14, 2024, 10:07 AM UTC

Duration: 15m 38s


  • Opened
    Jun 14, 2024, 10:07 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 14, 2024, 10:22 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 14, 2024, 10:06 AM UTC

Duration: 16m 34s


  • Opened
    Jun 14, 2024, 10:06 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 14, 2024, 10:23 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 14, 2024, 10:06 AM UTC

Duration: 16m 1s


  • Opened
    Jun 14, 2024, 10:06 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 14, 2024, 10:22 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 14, 2024, 10:05 AM UTC

Duration: 16m 48s


  • Opened
    Jun 14, 2024, 10:05 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 14, 2024, 10:22 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 14, 2024, 10:05 AM UTC

Duration: 17m 8s


  • Opened
    Jun 14, 2024, 10:05 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 14, 2024, 10:22 AM UTC

    Issue resolved on monitor recovery.

June 11, 2024

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 8:29 PM UTC

Duration: 7m 58s


  • Opened
    Jun 11, 2024, 8:29 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 8:37 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 8:28 PM UTC

Duration: 8m 16s


  • Opened
    Jun 11, 2024, 8:28 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 8:36 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 8:28 PM UTC

Duration: 8m 31s


  • Opened
    Jun 11, 2024, 8:28 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 8:37 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 8:27 PM UTC

Duration: 9m 24s


  • Opened
    Jun 11, 2024, 8:27 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 8:36 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 8:25 PM UTC

Duration: 9m 55s


  • Opened
    Jun 11, 2024, 8:25 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 8:35 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 8:25 PM UTC

Duration: 10m 2s


  • Opened
    Jun 11, 2024, 8:25 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 8:35 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 8:20 PM UTC

Duration: 16m 3s


  • Opened
    Jun 11, 2024, 8:20 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 8:36 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 8:20 PM UTC

Duration: 16m 45s


  • Opened
    Jun 11, 2024, 8:20 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 8:37 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 12:31 PM UTC

Duration: 7m 2s


  • Opened
    Jun 11, 2024, 12:31 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 12:38 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 12:31 PM UTC

Duration: 7m 30s


  • Opened
    Jun 11, 2024, 12:31 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 12:38 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 12:30 PM UTC

Duration: 4m 32s


  • Opened
    Jun 11, 2024, 12:30 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 12:35 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 12:30 PM UTC

Duration: 10m 48s


  • Opened
    Jun 11, 2024, 12:30 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 12:41 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 12:30 PM UTC

Duration: 10m 44s


  • Opened
    Jun 11, 2024, 12:30 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 12:41 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 12:29 PM UTC

Duration: 5m 37s


  • Opened
    Jun 11, 2024, 12:29 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 12:34 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 11, 2024, 12:22 PM UTC

Duration: 13m 40s


  • Opened
    Jun 11, 2024, 12:22 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 11, 2024, 12:36 PM UTC

    Issue resolved on monitor recovery.

June 7, 2024

Healthcheck Assertion Failed

Outage

Opened: Jun 7, 2024, 3:04 AM UTC

Duration: 39s


  • Opened
    Jun 7, 2024, 3:04 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 7, 2024, 3:04 AM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 7, 2024, 3:04 AM UTC

Duration: 43s


  • Opened
    Jun 7, 2024, 3:04 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 7, 2024, 3:05 AM UTC

    Issue resolved on monitor recovery.

June 6, 2024

Site Unreachable

Outage

Opened: Jun 6, 2024, 12:18 PM UTC

Duration: 41m 58s


  • Opened
    Jun 6, 2024, 12:18 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 6, 2024, 12:37 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jun 6, 2024, 12:48 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jun 6, 2024, 12:58 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jun 6, 2024, 1:49 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jun 6, 2024, 2:03 PM UTC

    Issue resolved on monitor recovery.

June 5, 2024

Site Unreachable

Outage

Opened: Jun 5, 2024, 3:53 PM UTC

Duration: 36m 59s


  • Opened
    Jun 5, 2024, 3:53 PM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 5, 2024, 3:55 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jun 5, 2024, 4:03 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jun 5, 2024, 4:29 PM UTC

    Issue resolved on monitor recovery.

  • Unresolved
    Jun 5, 2024, 5:12 PM UTC

    Issue re-opened on monitor failure.

  • Resolved
    Jun 5, 2024, 5:20 PM UTC

    Issue resolved on monitor recovery.

Healthcheck Assertion Failed

Outage

Opened: Jun 5, 2024, 10:50 AM UTC

Duration: 4m 10s


  • Opened
    Jun 5, 2024, 10:50 AM UTC

    Issue opened on monitor failure.

  • Resolved
    Jun 5, 2024, 10:55 AM UTC

    Issue resolved on monitor recovery.

May 27, 2024

Healthcheck Assertion Failed

Outage

Opened: May 27, 2024, 2:29 PM UTC

Duration: 10m 53s


  • Opened
    May 27, 2024, 2:29 PM UTC

    Issue opened on monitor failure.

  • Resolved
    May 27, 2024, 2:40 PM UTC

    Issue resolved on monitor recovery.

May 17, 2024

Healthcheck Assertion Failed

Outage

Opened: May 17, 2024, 2:08 PM UTC

Duration: 271h 51m 15s


  • Opened
    May 17, 2024, 2:08 PM UTC

    Issue opened on monitor failure.

  • Resolved
    May 28, 2024, 9:59 PM UTC

    Issue resolved on monitor recovery.

May 15, 2024

Site Unreachable

Outage

Opened: May 15, 2024, 3:48 PM UTC

Duration: 3m 51s


  • Opened
    May 15, 2024, 3:48 PM UTC

    Issue opened on monitor failure.

  • Resolved
    May 15, 2024, 3:52 PM UTC

    Issue resolved on monitor recovery.

Site Unreachable

Outage

Opened: May 15, 2024, 2:37 PM UTC

Duration: 4m 57s


  • Opened
    May 15, 2024, 2:37 PM UTC

    Issue opened on monitor failure.

  • Resolved
    May 15, 2024, 2:42 PM UTC

    Issue resolved on monitor recovery.

May 14, 2024

Healthcheck Assertion Failed

Outage

Opened: May 14, 2024, 1:54 PM UTC

Duration: 5m 44s


  • Opened
    May 14, 2024, 1:54 PM UTC

    Issue opened on monitor failure.

  • Resolved
    May 14, 2024, 2:00 PM UTC

    Issue resolved on monitor recovery.

May 8, 2024

Healthcheck Assertion Failed

Outage

Opened: May 8, 2024, 2:04 PM UTC

Duration: 47h 1m 12s


  • Opened
    May 8, 2024, 2:04 PM UTC

    Issue opened on monitor failure.

  • Resolved
    May 10, 2024, 1:05 PM UTC

    Issue resolved on monitor recovery.