Frame skips/reuses with most monitors?

[SubstantialCt8690]

Sorry if this is a bad place to ask, kind of out of ideas where else.

We're developing some VR hardware and using a monitor as the output display device during prototyping of our code. CRU has been a very valuable tool for this job by allowing to use non-standard refresh rates such as 90 (common in VR).

Occasionally I have a frame in a DisplayPort (DP) video stream being skipped at the monitor, and monitor displaying previous frame again. Tried with few monitors, the issue is present on 3, not present on 2 laptops with the builtin MIPI connected monitors but present on a 3rd one, and not present on a DisplayPort-connected consumer VR headset display but which has a closed-source PC runtime and PCB we can't analyze and learn from.
Happens at 60-240Hz.

Initially I thought something is wrong with out graphics code, but after trying many modes and settings in the code I'm starting to suspect the issue is the signal integrity at the cables or at the monitors causing a frame to sometimes be degraded enough to not be display-able. Once we present the frames to the graphics API, we can't ask the GPU to tell us what's going on with the frame and whether it reached the DP port successfully and on time or not. So I can't tell if it's the GPU skipping the current frame and sending the previous one for some reason, or if it's the Monitor displaying the previous frame from its own PCB framebuffer for some reason.

I could get some more monitors or better shielded DP cables or a DP booster device and run more tests, but I think a more valuable and reliable data right now would be one from a DP signal analyzer or tester either connected instead of the monitor, or sitting between the PC and the monitor, to be sure if the issue is caused by the PC motherboard or the cable used or not.

I don't need much functionality from the DP analyzer device other than detecing dropped/degraded frames and logging it to preferrably be accessible from the PC. Need it to handle up to 1080p at 240Hz, or at least 1080p at 180Hz.

Does anyone here have any experience with such devices? Which would you recommend for our specific requirement?

Finally, do you think a video created by a video capture card will be good enough for such tests? My fear is they may resample the frames so 1 frame IN does not always mean 1 frame OUT, even when input and output fps remains the same.

Thanks

[ToastyX]

How often is occasionally? How are you detecting the frame skips? I've only seen frame skipping on the monitor's end when it's doing frame rate conversion on an unsupported refresh rate, like for 90 Hz it would skip every third frame to convert to 60 Hz, but that would be obvious, and what you're talking about is the frame repeating instead of displaying the new frame, which is different.

I haven't heard of cables causing frame skipping because usually transmission errors would appear as artifacts, or the screen would blank out if the signal was really bad. The monitor shouldn't ignore a whole frame just because of a minor error, and in fact that wouldn't be possible unless the monitor buffered the entire frame before displaying it, which would delay the output by one frame and cause lag. There are monitors that buffer frames, but low-lag gaming monitors and cheap monitors that support more than 60 Hz generally don't.

If you can prove certain monitors are repeating a frame every once in a while, that would be interesting information. There were some 240 Hz monitors that had a frame skipping bug in the firmware with certain timing parameters, but that was skipping, not repeating like you described: https://forums.blurbusters.com/viewtopic.php?t=3598

I don't have any experience with DisplayPort analyzer devices, but I have an interesting idea. If you can display the hardware mouse cursor with your output, have a separate background program that constantly moves the mouse cursor while testing the output, or use a 1000 Hz mouse and move the mouse manually in circles. The hardware mouse cursor normally doesn't skip frames since it's handled separately by the GPU, so if the cursor moved but the frame is the same, you know for sure it's on the PC's end.

[SubstantialCt8690]

Thanks so much for your response. Sorry for the delay, I had to collect the needed data.

(10-05-2023 10:43 PM)ToastyX Wrote:  How often is occasionally?

Once every few seconds.

Quote:How are you detecting the frame skips?

Just analyzing video recording and seeing two red frames displayed in sequence rather than blue-red the moment the artifact happens. That said, camera fps may be too low for this.

Quote:I've only seen frame skipping on the monitor's end when it's doing frame rate conversion on an unsupported refresh rate, like for 90 Hz it would skip every third frame to convert to 60 Hz

That would be a more cosistent artifact, but in our case it happens once every few seconds and is more random, there's no fixed time intervals.

Quote:frame repeating instead of displaying the new frame

Technically it could be displaying the new frame instead of repeating the old frame, I could be wrong. I've added a frame counter to the latest videos where the mouse position is also controlled by code.
The reason I don't think the next frame is being displayed instead of frame being repeated, is because I think the next frame being displayed automatically implies one frame has been skipped, and I don't see a black frame insertion in the video recordings. Unless somehow a frame could be skipped without delaying the frame coming after it.

1) I should have specified it earlier, my main test monitor is "AOC C27G2Z".
I've tested with two laptops (Asus ROG G752VS and Aorus 15G XC-8US2430SH ). The programmer I work with has also tested on same monitor and his PC and his laptop (RTX4080). We've also tested on the display panels of our laptops and all but two (but not all 3) laptop displays also have this issue. The programmer has also tested on these monitors with the same results: Monster Aryond A32 V1.3 165 Hz, ASUS vs239, Samsung Odyssey Neo G7 LS32BG750NUXUF. All have this issue.
For AOC C27G2Z monitor and AORUS 15G XC-8US2430SH laptop, I've tested both with DisplayPort and HDMI.
So to summarize, only one laptop's display ( Aorus 15G XC-8US2430SH ) does not have this issue at 240Hz.

Here's a screenshot of the current CRU settings for the AOC monitor:



Note that I've also tested with the new refresh rates specified both in the "Detail Resolutions" slots and the "Extension Blocks", it seems to make no difference in the results.

Here's the code and builds of the test programs. There's three. One was written in pure DirectX11, another one using the Unity game engine and a third one using Panda3D game engine which uses OpenGL 2: https://e.pcloud.link/publink/show?code=...xeN53JngMk

2) Following the ideas from your linked Blurbusters forum topic, I tried both (a) changing the refresh rate from 240 to 239.9 and 239, and also tweaked the "Back porch" number by decreasing it by 1. I see no difference. I think you may be on to something here, but I can't guess the correct values.

This is what we see (epilepsy warning) (yes, it appears to my eyes exactly as in the videos): https://e.pcloud.link/publink/show?code=...VYFbuP7vwk

3) The reason I consider that it is the frame being reused rather than frame merely skipped, is because we see a bright flash rather than a brief moment of blackness. Flash could be explained by the red wavelengths appearing brighter to the human eye than blue: https://www.giangrandi.org/optics/eye/slf-log-small.png
So when you see two red frames in a sequence rather than the usual red/blue sequence, your eye notices something brighter for a brief moment (extra red frame instead of blue).
Also, I've made some 60Hz camera recordings when the monitor itself was set to run at 60Hz, and that's what I see (repeating frames) when analyzing the captured frames in a video editor (link to these videos are above).

4) As suggested, here's a program controlling the mouse movement while also changing the color each frame: https://e.pcloud.link/publink/show?code=...TQyHINiiik

These are my results with it: https://e.pcloud.link/publink/show?code=...33jyTiCWy7

I don't think my 120fps phone camera is good enough for capturing 120 and 240Hz results without blending of the frame.
I do notice the mouse cursor skipping a position once in a while even at 60Hz, not sure if it's a camera capture artifact or real.
For example in "60Hz - AOC C27G2Z.mp4", when frame counter shows 977, then 978, the mouse cursor remains in the same place on location 2. Then when it shows 979, the mouse cursor jumps straight to location 4. Same happens when the counter is at 1038. It's hard to tell at 60Hz when the artifact happens because everything is already flickering.

Besides this, I can try getting a higher fps camera.

[ToastyX]

This is not a good way to test what's happening. Since you're using alternating colors, frame skipping and frame repeating would look exactly the same because it would display the same color twice in both cases, and the mouse test isn't being done correctly. I emphasized using a background program to constantly move the mouse cursor, ideally multiple times per refresh to ensure it doesn't miss a refresh. Moving the mouse cursor in the same program once per refresh is unreliable when the program is potentially missing refreshes. Adding a frame counter helps a little, but I'm having trouble locating frame skips in the videos because the camera is not perfectly synchronized with the monitor, and every video player I've tried is having trouble synchronizing 120 FPS videos with my monitor even at 120 Hz, so it doesn't even look the same on consecutive views. When it does happen to synchronize correctly, I can't locate any frame skips in the 60 Hz and 120 Hz videos, and the 240 Hz video is no good because the video is 120 FPS. Let me know if you have specific frame numbers that show the frame skipping or repeating, not just the mouse cursor.

BlurBusters has a frame skipping test here: https://testufo.com/frameskipping

You need to design something like that so you can easily differentiate between frames while at the same time having another process running in the background to move the mouse cursor every millisecond in a straight line at a consistent speed so you can easily see skips. Then all you need to do is take pictures with a slow enough shutter speed to capture multiple squares. If you see a brighter square, then it's frame doubling. If you see a blank square, then it's frame skipping. Then you can cross reference with the mouse cursor to see if it corresponds.

I tried the programs directly and they actually work quite well on my setup, rarely any frame skips. When I do see a frame skip, the frame counter dips, which indicates it's happening on my PC's end. Keep in mind Windows is not a real-time operating system, so there's no way to guarantee the programs won't miss a refresh every once in a while, especially at 240 Hz where the refresh time is around 4 ms, so a background process can easily cause a missed refresh.

[SubstantialCt8690]

I just found an Exilim camera I already had left from another project which can record at low resolution at 480Hz. I can use that instead.

Sure, I can modify the test program to work better. But help me understand few points first.

(10-11-2023 01:10 PM)ToastyX Wrote:  This is not a good way to test what's happening. Since you're using alternating colors, frame skipping and frame repeating would look exactly the same because it would display the same color twice in both cases

Are you saying missing a frame won't result in black frame insertion for the frame the monitor didn't receive a data to display? Will it hold/reuse the previous frame?

Quote:and the mouse test isn't being done correctly. I emphasized using a background program to constantly move the mouse cursor, ideally multiple times per refresh to ensure it doesn't miss a refresh. Moving the mouse cursor in the same program once per refresh is unreliable when the program is potentially missing refreshes.

My concern wasn't that the program is missing frames (since if the frames are dumped as files, all seems well there), but the entire DisplayPort/HDMI stream or monitor is, since we tried with different game engines and pure graphics APIs.
I can try that too, we don't need a separate program still, most engines allow to run functionality at either each refresh or a specific needed time duration which can be shorter than the frame duration (such as 1ms).

Quote:BlurBusters has a frame skipping test here: https://testufo.com/frameskipping

I can recreate that since we still need something like a red-blue alternating cards next to it so that we can visually detect the artifact with our own eyes and know where to look for the issue in the captured video.

Quote:I tried the programs directly and they actually work quite well on my setup, rarely any frame skips. When I do see a frame skip, the frame counter dips, which indicates it's happening on my PC's end. Keep in mind Windows is not a real-time operating system, so there's no way to guarantee the programs won't miss a refresh every once in a while, especially at 240 Hz where the refresh time is around 4 ms, so a background process can easily cause a missed refresh.

Sure, in our tests nothing is running in the background.

Windows is used for VR which is as realtime as you can expect. For this NVidia provides a mode to control external monitor without the OS interfering, called DirectMode. We have a test runtime that can make any monitor function as a SteamVR headset display, with the addition of a registry key. The thing is, the frame skips happen even when doing so. But other SteamVR headsets don't seem to have this issue. So I suspect it's us or the monitor doing something wrong.
Sure, I don't mean Windows will never cause artifacs like this, but in our case it's happening once every few seconds, unlike the other VR headsets.

If you're okay with sharing this info, what monitor model and refresh rate did you test on your end?

I will send updated data asap.

[ToastyX]

(10-11-2023 04:41 PM)SubstantialCt8690 Wrote:  Are you saying missing a frame won't result in black frame insertion for the frame the monitor didn't receive a data to display? Will it hold/reuse the previous frame?

The monitor always receives data to display at a consistent rate. The signal wouldn't be correct without any data, and the monitor would assume there is no signal. The GPU has a frame buffer that is constantly streamed to the monitor every refresh with whatever data is already in it, and that's what the program is affecting. Actually it's double-buffered, so the program writes to the back buffer which is then swapped with the front buffer after presenting, and if synchronizing to vertical blank (vsync), the swap is delayed until the next vertical blanking period between refreshes. If not, it will tear if presenting during a refresh because the GPU already sent part of the last frame to the monitor, so the monitor will display part of the old frame and part of the new frame. In the case of G-SYNC/FreeSync, the vertical blanking period is dynamically extended to delay the next refresh until a new frame is ready, and the current frame is held on the monitor until the next refresh.

(10-11-2023 04:41 PM)SubstantialCt8690 Wrote:  Sure, in our tests nothing is running in the background.

Windows is used for VR which is as realtime as you can expect. For this NVidia provides a mode to control external monitor without the OS interfering, called DirectMode. We have a test runtime that can make any monitor function as a SteamVR headset display, with the addition of a registry key. The thing is, the frame skips happen even when doing so. But other SteamVR headsets don't seem to have this issue. So I suspect it's us or the monitor doing something wrong.
Sure, I don't mean Windows will never cause artifacs like this, but in our case it's happening once every few seconds, unlike the other VR headsets.

Windows is always doing stuff in the background unfortunately. I bet if you made a DOS program for testing vsync, the output would be perfect, but I don't know if that's feasible for you. So far I'm not seeing anything that makes me think this is a monitor issue. This looks exactly like the program is missing refreshes for whatever reason, which may not be the program's fault.

I can think of some things to check:

1. Try using the multimedia timer API to set the timer resolution to 1 ms. This changes globally when any program requests it, so this is already happening on my end for whatever reason. You can use this to check: https://www.lucashale.com/timerresolutio...on_1.2.zip

2. Use LatencyMon to see if there are any DPC latency issues: https://www.sweetwater.com/sweetcare/art...cy-issues/

3. Try starting the program with high priority and see if that changes the results: https://superuser.com/questions/699651/s...h-priority

4. If any of the monitors have G-SYNC/FreeSync, try with and without G-SYNC/FreeSync enabled.

(10-11-2023 04:41 PM)SubstantialCt8690 Wrote:  If you're okay with sharing this info, what monitor model and refresh rate did you test on your end?

Acer XV272U Pbmiiprzx. I tried 144 Hz, 120 Hz, and 60 Hz.

[SubstantialCt8690]

Okay. I'm still preparing the data to share.

(10-11-2023 07:13 PM)ToastyX Wrote:  and the current frame is held on the monitor until the next refresh.

I think this is the part I needed to know.
It seems like even if I disable Freesync on the monitor but enable vsync in code, this still happens. Is this expected?

(10-11-2023 04:41 PM)SubstantialCt8690 Wrote:  Windows is always doing stuff in the background unfortunately.

Sure, by "nothing" I mean we are not explicitly running anything in the background ourselves which may affect it. Like I said, it doesn't seem to be an issue for other VR headsets, unless there's very heavy background load.

Quote:So far I'm not seeing anything that makes me think this is a monitor issue.

To clarify one point, the Aorus laptop's own 240Hz display does not have this artifact running the same code, while the same laptop connected to the AOC 240Hz monitor running the exact same code produces the artifact. Now that you've explained that monitors can skip frames, hold onto the frame when they do that, and linked to a forum topic detailing how certain monitors skip frames out of the box, I'm not sure why you're not thinking it's not a monitor issue. I can understand if you mean you have no definitive proof that it is.

Quote:I can think of some things to check:

1. Try using the multimedia timer API to set the timer resolution to 1 ms. This changes globally when any program requests it, so this is already happening on my end for whatever reason. You can use this to check: https://www.lucashale.com/timerresolutio...on_1.2.zip

2. Use LatencyMon to see if there are any DPC latency issues: https://www.sweetwater.com/sweetcare/art...cy-issues/

3. Try starting the program with high priority and see if that changes the results: https://superuser.com/questions/699651/s...h-priority

Let us go through these and come back to you.

Quote:4. If any of the monitors have G-SYNC/FreeSync, try with and without G-SYNC/FreeSync enabled.

Done already with the main AOC 240Hz monitor.

Thanks

[ToastyX]

(10-11-2023 08:17 PM)SubstantialCt8690 Wrote:  

(10-11-2023 07:13 PM)ToastyX Wrote:  and the current frame is held on the monitor until the next refresh.

I think this is the part I needed to know.
It seems like even if I disable Freesync on the monitor but enable vsync in code, this still happens. Is this expected?

You're misunderstanding what I'm saying. LCD monitors are sample and hold displays. The pixels stay lit until the next refresh, unlike CRTs where the image starts to fade immediately, so what I described always happens. G-SYNC/FreeSync just increases the vertical blanking period to delay the next refresh, so the pixels stay lit during that time.

That's not to say the monitor can't have a buffer of its own, but monitors that don't delay frames basically just stream the incoming image straight to the pixels, so for it to skip a frame and just display the last frame again, it would have to ignore the next incoming frame entirely for some reason, which I've mentioned I've only seen happen when doing frame rate conversions from unsupported refresh rates.

Actually you just gave me an idea. CRT monitors are directly driven by the signal and would not buffer or skip frames. If you can get the issue to happen with a CRT monitor, it would be impossible for the problem to be the monitor. The problem is modern GPUs don't have VGA output, and using an active adapter would just add another variable to the mix, so you'd need an NVIDIA 900-series or older GPU if you want to avoid active adapters.

(10-11-2023 04:41 PM)SubstantialCt8690 Wrote:  To clarify one point, the Aorus laptop's own 240Hz display does not have this artifact running the same code, while the same laptop connected to the AOC 240Hz monitor running the exact same code produces the artifact. Now that you've explained that monitors can skip frames, hold onto the frame when they do that, and linked to a forum topic detailing how certain monitors skip frames out of the box, I'm not sure why you're not thinking it's not a monitor issue. I can understand if you mean you have no definitive proof that it is.

You're saying this is happening with several different monitors, which would mean the problem would have to be widespread, so I'm trying to eliminate as many factors as possible. Frame skipping in monitors is consistent and periodic, while you described random intervals, which is more likely to happen on the PC's end. The frame skipping issue with certain 240 Hz monitors is a firmware bug and not a common problem, and that was also consistent and periodic.

I should also mention that laptop displays normally don't do any scaling or special processing, so they usually don't skip frames, but you mentioned one laptop had the problem. It would be interested to know if that laptop uses TMDS or eDP internally.

I just realized, when connecting a monitor to a laptop, you end up with two displays. Historically, Windows had issues with multiple displays and would only synchronize with one display, causing stutter on the other, so ideally you should disable the laptop screen when testing the external monitor.

Also, laptops with Intel CPUs and NVIDIA GPUs usually have switchable graphics, which may behave differently from systems with a dedicated GPU, but you mentioned the problem happening in both cases. What happens when using just the Intel GPU with the NVIDIA GPU disabled? Have you tested any AMD/ATI GPUs?

Another factor is the laptop screen and the external monitor might be using different timing parameters. Normally this shouldn't matter, but if there is a frame timing issue somewhere (not necessarily in your code but possibly a graphics driver or hardware bug), then it's possible certain timing parameters might work better than others. It would be interesting to see if you can reproduce the problem with one display and not the other on the same system if they were using the same exact timing parameters (including resolution and refresh rate).

Also, people on the Blur Busters forum are interested in this sort of thing and might have additional insight, so you might want to post there as well: https://forums.blurbusters.com/

[SubstantialCt8690]

(10-11-2023 10:26 PM)ToastyX Wrote:  for it to skip a frame and just display the last frame again, it would have to ignore the next incoming frame entirely for some reason, which I've mentioned I've only seen happen when doing frame rate conversions from unsupported refresh rates.

Sorry if you already explain this, but would it display the same frame again, or display black during that period? I don't know if sample and hold applies in this situation, all I know the pixel data is held until the next frame, but I don't know if that also applies when the next frame is skipped.

Quote:Actually you just gave me an idea. CRT monitors are directly driven by the signal and would not buffer or skip frames. If you can get the issue to happen with a CRT monitor, it would be impossible for the problem to be the monitor. The problem is modern GPUs don't have VGA output, and using an active adapter would just add another variable to the mix, so you'd need an NVIDIA 900-series or older GPU if you want to avoid active adapters.

Tricky test, requires buying old equipment, let's get back to this when/if all other tests are unsuccessful. Probably the least hassle way would be getting a GTX980 laptop. Thanks for mentioning this.

(10-11-2023 04:41 PM)SubstantialCt8690 Wrote:  You're saying this is happening with several different monitors, which would mean the problem would have to be widespread

Sure, but I guess 240Hz monitors aren't widespread and noticing a skip at 240Hz in a more normal use case may not be noticeable like here.

Quote:Frame skipping in monitors is consistent and periodic, while you described random intervals, which is more likely to happen on the PC's end. The frame skipping issue with certain 240 Hz monitors is a firmware bug and not a common problem, and that was also consistent and periodic.

I hear you, but I'm just wondering if there may be other cases when frame skipping may happen besides the listed ones, when it doesn't have to be periodit. Like signal integrity issues at very high refresh rates.

Quote:I should also mention that laptop displays normally don't do any scaling or special processing, so they usually don't skip frames, but you mentioned one laptop had the problem. It would be interested to know if that laptop uses TMDS or eDP internally.

It's an ASUS G752VS. I'm not sure how I could find out that information.
Is it possible that I've simply messed up its settings with CustomResolutionUtility? Maybe I've let CRU change the blanking numbers to some calculated numbers? Can CRU even do that?

Quote:I just realized, when connecting a monitor to a laptop, you end up with two displays. Historically, Windows had issues with multiple displays and would only synchronize with one display, causing stutter on the other, so ideally you should disable the laptop screen when testing the external monitor.

Good idea, I'll try that and report back. Definitely also happens with the programmer's desktop PC and main monitor.

Quote:What happens when using just the Intel GPU with the NVIDIA GPU disabled? Have you tested any AMD/ATI GPUs?

I'll try with Intel GPU (although I suspect it won't even be able to handle smooth 240Hz or even let me set above 60Hz). We don't have access to AMD GPUs.

Quote:t would be interesting to see if you can reproduce the problem with one display and not the other on the same system if they were using the same exact timing parameters (including resolution and refresh rate).

Yes, that's exactly what happens with the Aorus latop, internal 240Hz 1080p display doesn't have the artifact, the connected external AOC 240Hz 1080p monitor has.

Quote:Also, people on the Blur Busters forum are interested in this sort of thing and might have additional insight, so you might want to post there as well: https://forums.blurbusters.com/

Great, thanks. I'll supply the promised data (including the proper mouse cursor test program and 480Hz recorded videos) here before sharing the topic with them.

[ToastyX]

(10-11-2023 11:10 PM)SubstantialCt8690 Wrote:  

(10-11-2023 10:26 PM)ToastyX Wrote:  for it to skip a frame and just display the last frame again, it would have to ignore the next incoming frame entirely for some reason, which I've mentioned I've only seen happen when doing frame rate conversions from unsupported refresh rates.

Sorry if you already explain this, but would it display the same frame again, or display black during that period? I don't know if sample and hold applies in this situation, all I know the pixel data is held until the next frame, but I don't know if that also applies when the next frame is skipped.

Actually, I got mixed up because skipping a frame and displaying the last frame again would be both frame skipping and frame repeating, depending on the point of view. It's skipping a frame of input and repeating a frame of output, which is something I haven't seen a display do. When doing frame rate conversions, it does either one or the other, but not both. Regardless, it would not display black unless you send it black.

If it's converting 120 Hz input to 60 Hz output, it will skip every other frame of input and send 60 FPS to the panel, so from the panel's point of view it's only operating at 60 Hz and displaying each frame once. Here is an example of a monitor that did exactly that: https://www.monitortests.com/monoprice-2...-large.jpg

If it's converting from 30 Hz input to 60 Hz output, it won't skip any frames of input, and it will send each frame to the panel twice, so from the panel's point of view, it's operating at 60 Hz but displaying each frame twice.

Actually to throw more confusion into the mix, there are monitors that can strobe the backlight to reduce motion blur, but it wouldn't be enabled by default because you normally have to explicitly enable this feature. In this case, the backlight only flashes briefly after finishing a refresh and stays black the rest of the time, but that only affects what you see, not how the panel actually refreshes, which doesn't change.

(10-11-2023 04:41 PM)SubstantialCt8690 Wrote:  Is it possible that I've simply messed up its settings with CustomResolutionUtility? Maybe I've let CRU change the blanking numbers to some calculated numbers? Can CRU even do that?

CRU doesn't change anything on the display's end, so running reset-all.exe and rebooting would reset everything back to default.