Hidden cursor can have an action

[erikos]

follow up on #4021

When the cursor is hidden because we are in 'touch mode' it can still influence the UI, when over a toolbar icon have the hover background or similar.

[garycmartin]

Testing in 13.1.0 build 8, I'm finding Terminal a frustratingly good example of this again just now, it's still not 100% reproducible but happens when I least want it to ;)

To reproduce:

1) using Terminal in ebook mode
2) reveal the Frame with a down swipe in the toolbar
3) tap the Home icon to get back to the Home zoom view

Result:

As the frame slides away, the mouse cursor must be floating over Terminal Toolbar View icon, the zoom view briefly appears, then the Terminal View secondary toolbar hover must have triggered and switches you back to Terminal with the View secondary toolbar now in an open hover state. Just managed to trigger this 4 times in a row trying to get back to home, now after switching to a different view and back to home from there its behaving again.

You can also see this if you have several activities open with plenty of tools (e.g. Calculate, Turtle Art). Using the Frame to switch activities, will leave the invisible cursor hovering over the toolbar, so after you switch, a moment later a toolbar help hint will suddenly reveal itself. Actually this is a more reliable test case, happens every time here.

[dsd]

There is some weird interaction with the mouse cursor and the touchscreen.

In Sugar, playing on an empty group view (I have no friends), I do the following:

1. As preparation, use the touchpad to position the mouse cursor somewhere on the left half of the screen
2. Briefly touch somewhere on the right side of the screen
3. Use the touchpad to move the cursor very slightly
4. Briefly touch somewhere on the left side of the screen
5. Use the touchpad to move the cursor very slightly
6. Repeat steps 2 through 5 until bored

The result is that touching the screen always moves the mouse cursor to the location of the touch (you can't see it, but move the mouse slightly and you'll find its new location).

Now, in GNOME:

1. Start GNOME, wait for it to load, don't touch any input devices. Note that the mouse pointer is in the centre of the screen.
2. Touch the screen somewhere on the right hand side.
3. Use the touchpad to move the cursor very slightly

You'll notice that the mouse cursor jumps to the touch point when you execute number 3, just like Sugar. But now keep playing:

1. Touch the screen somewhere on the left side
2. Use the touchpad to move the cursor very slightly
3. Repeat 1 and 2 until bored

Now we are seeing different behaviour. Touching the screen does *not* affect the position of the mouse cursor like the previous tests.

But when you open a GNOME app or two and start tapping in and out of windows, we lose that behaviour.

Having trouble getting my head around what kind of behaviour we want here, and if it relates to this bug.

[garnacho]

FWIW, I think the described pointer jumpyness might be fixed by the patches in ​http://lists.x.org/archives/xorg-devel/2012-October/034043.html, as per Peter's recent response, ​http://lists.x.org/archives/xorg-devel/2012-November/034369.html

[erikos]

A very good test case is the following:

Build: os11

Steps to reproduce:

• open Journal

• go to detail view of an entry

• go back to list view

---> the favourite icon of that entry is coloured and has a grey background, like when hovered

If you click the back button in the detail view not at the far left the text entry could be focused which is an indication that this is triggered by the cursor.

[manuq]

Another testcase:

• start Browse and navigate olpc library: Cience > Biology
• scroll with the finger

the rows at the initial point the scroll gesture was started get hover, are highlighted.

[dsd]

This package includes the X patches mentioned by Carlos: ​http://dev.laptop.org/~dsd/20121114/

Doesn't seem to help in my quick test.

Complete list of touch patches included:

Applying: Sync TouchListener memory allocation with population in TouchSetupListeners()
Applying: Xi: Don't check for TOUCH_END, it's never set
Applying: Xi: don't deliver TouchEnd to a client waiting for TouchBegin (#55738)
Applying: Xi: Set modifier mask on touch events
Applying: Xi: Update the device after delivering the emulated pointer event(#56558)
Applying: remove init_event
Applying: Update the MD's position when a touch event is received
Applying: Don't use GetTouchEvents when replaying events
Applying: Don't use GetTouchEvents in EmitTouchEnd
Applying: Simplify GetTouchEvents

[erikos]

Immediate priority for the next days.

[dsd]

I've been trying to get an understanding of this:

​http://lists.x.org/archives/xorg/2012-November/055070.html

But I think I have not managed to communicate the problem well enough, and I don't have enough X knowledge to clearly understand the responses.

My next steps were to write a GTK+ app to show the behaviour that closing one window would activate hover on a widget in the window below. i.e. an app opens two windows, one on top of the other. The bottom window has a widget that provides a clear visual response when the mouse is hovering over it. The top window has a button, when clicked the window is hidden. Here is my start: ​http://dev.laptop.org/~dsd/20121127/test.c

That would then have to be ported to Xlib to provide a really good test case for Peter H.

Or maybe Carlos will take a more direct approach to investigating/fixing with his good X knowledge.

[garnacho]

GTK+ patch

[garnacho]

This is pretty much a legit behavior from Xorg, what happens here is that as the frame window moves, a crossing event is generated by Xorg when the pointer gets to be on the new window, but that event is not generated by an input device, and hence it has both deviceid and sourceid set to the master device, this fools gdk as it uses the source device to find out whether crossing event filtering or synthesizing must happen.

with this patch all clients get to know the last device the user interacted with, so can prepare in advance to getting the pointer on them

[erikos]

patched gtk3 rpms: ​http://dev.laptop.org/~erikos/result_gtk3/

For showing the cursor again do:

edit /usr/bin/sugar-session

comment: GObject.idle_add(setup_cursortracker_cb)

[erikos]

For reproducing ​http://bugs.sugarlabs.org/ticket/4068#comment:1 you have to launch the Terminal activity from the Home screen with a touch event. Then I can trigger the behaviour reliably. I can reproduce as well when the cursor is not hidden.

[erikos]

More explanation bits:

garnacho_: crossing events happens when the pointer crosses from one window to another, there's two ways to get that happen, moving the pointer, or moving the windows

garnacho_: for the first case gtk+ knows the hw device that made that happen, and it's what gtk+ uses to behave sensible on touchscreens

garnacho_: on the second case, xorg itself tells nothing about the hw device, because no user input generated the event this time

garnacho_: and gtk+ doesn't deal with that as fine

[garnacho]

gtk+ patch

[garnacho]

This new patch replaces the previous one, turns out gdk already made some work into hiding those sourceid!=slave kind of events that didn't make that patch as effective

[erikos]

Looking good in longer term testing here. It does solve ​http://bugs.sugarlabs.org/ticket/4068#comment:1

Daniel if you agree I can pull the rpm in.

[erikos]

Rpms for testing can be found here: ​http://dev.laptop.org/~erikos/result_gtk3_2/

The test case described in #comment:4 is actually not a side effect of this one to my understanding. The following test should indicate this:

• go to the detail view of the third entry in the list

• then use the back button to go back to the detail view

---> the favi icon of the third entry is highlighted (but the finger pushed the back button which had similar coordinates then the favi icon of the first entry)

[manuq]

Good, I can't reproduce the undesired switch to the previous activity in comment 1, Gary's first testcase. I can still reproduce the second:

1. open Calculate, which has many subtoolbars
2. do the swipe to open the frame over one of the first icons, the ones with the down arrow
3. wait

You can see the subtoolbar appears below the frame.

Replying to erikos:

Rpms for testing can be found here: ​http://dev.laptop.org/~erikos/result_gtk3_2/

The test case described in #comment:4 is actually not a side effect of this one to my understanding.

But do you agree this is not desireable? Another testcase: in the Journal list, you can see the hovering issue scrolling by touch vertically over the favs or activity icon columns. You will see the icons getting hover (grey background) state. Which is the same as the Browse hover issue in comment 5. I can also reproduce dsd testcase in comment 2.

[erikos]

Replying to manuq:

Good, I can't reproduce the undesired switch to the previous activity in comment 1, Gary's first testcase. I can still reproduce the second:

1. open Calculate, which has many subtoolbars
2. do the swipe to open the frame over one of the first icons, the ones with the down arrow
3. wait

You can see the subtoolbar appears below the frame.

This is actually a good proof of Carlos' patch in place. This does only happen to GTK+2 activities (e.g. Calculate, Moon, Memorize...). The GTK+3 activities do work (e.g. Write, Browse...).

[erikos]

Manuel reports one regression with this, we compare two machines, both have the 'cursor-hiding-for-touch' disabled so we can see where the cursor is. Machine A has the latest rpm with the patch from Carlos.

TestCase:

• go to home view
• move the trackpad a bit so that the cursor is there for sure
• use touch to reveal the Frame and hide it again

On machine A (with the patch) the cursor disappears when hiding the Frame.

On machine B the cursor stays at the same position when hiding and showing the Frame.

[erikos]

I see again even stranger results, XO-4 with the patch applied:

• start machine

---> cursor visible

• use the view keys to switch between views

---> cursor stays visible

• open Browse

• click into the google search in the OLPC landing page

• move so that the cursor is visible

• use the view keys to switch between views

---> cursor disappears

Now we seem to have introduced a weird cursor behaviour. When I go to the Journal I can hide the cursor sometimes by moving over specific areas. Switching between views and activities does hide the cursor.

[manuq]

Replying to erikos:

I see again even stranger results, XO-4 with the patch applied:

• use the view keys to switch between views

---> cursor disappears

Yes I can reproduce here. Also cursor appears for an instant switching from a zoom view to browse and viceversa, but not between zoom views.

[manuq]

(wrote part of my comment in the description by mistake)

Replying to manuq:

Yes I can reproduce here. Also cursor appears for an instant switching from a zoom view to browse and viceversa, but not between zoom views.

I can also see the strange behaviour in Journal, cursor is hidden in some areas. Very difficult to define in which cases, is not consistent.

None of that happens without the patched rpms.

[garnacho]

Xorg patch

[garnacho]

The patch I've just attached fixes the bug described in comment 18. There are remaining Xorg issues with software rendered cursors where it is not redrawn when it's on a redrawn area itself, that's the cause for the weirdness described in comment 19, which I'm still investigating

[garnacho]

I investigated further this bug, so far I tracked the remaining issues down to an Xorg mishap with animated cursors (eg. the WATCH cursor type). It looks like Xorg fails to unset things internally when an animated cursor is running on a window that's destroyed as the animation runs. After that, the typical handlers that ensure the pointer is redrawn if on a redrawn area aren't further set.

This can be reproduced after closing i.e Write for the first time, the watch clock is set as the app closes, and leaves Xorg in this inconsistent state for anything after.

Hopefully today I'll come up with a patch for this

[garnacho]

Updated first xorg patch

[garnacho]

second patch, fixes pointer disappearing issues

[garnacho]

With these two last patches, all these remaining oddities are fixed, I'll also submit those upstream for discussion

[erikos]

rpms to test ​http://dev.laptop.org/~erikos/result_xorg_2/

[manuq]

Tested the rpms. The dissapearing cursor issues described in comments 18 and 19 above are gone. Nice!

[erikos]

Made a few more test and this looks really good now. Another test is:

• start Write

• close it with the cursor and do not move the mouse after closing

---

So we need 3 patches in total:

• gtk+ patch: ​http://bugs.sugarlabs.org/attachment/ticket/4068/0001-gdk-protect-better-against-crossing-events-with-sour.patch

• xorg 1. patch: ​http://bugs.sugarlabs.org/attachment/ticket/4068/0001-mi-Ensure-pointer-emulating-touch-events-update-the-.patch

• xorg 2. patch: ​http://bugs.sugarlabs.org/attachment/ticket/4068/0002-render-Unwrap-early-on-the-animated-cursor-BlockHand.patch

The main task is now to upstream those patches before we can include them. They should at least land in master.

[dsd]

The cursor hiding issue can also be seen on "unpatched" builds - i.e. maybe it is not linked to the changes that were made to solve the original issues on this bug. ​http://dev.laptop.org/ticket/12411

Anyway, the patch (0002-render-Unwrap-early-on-the-animated-cursor-BlockHand.patch) solves the issue. Thanks! Carlos, assuming its not a tricky job, can you prioritise upstreaming of this patch?

[dsd]

These 2 patches have been accepted by Peter H, thanks Carlos!

• [PATCH 1/2] mi: Ensure pointer emulating touch events update the sprite
• [PATCH 2/2] render: Unwrap early on the animated cursor BlockHandler

[erikos]

Another patch did land for this in master and the 3.6 branch ​https://git.gnome.org/browse/gtk+/commit/?h=gtk-3-6&id=63998cc93f6d8c10551b20f458b800471d6daf03

upstream bug: ​https://bugzilla.gnome.org/show_bug.cgi?id=691572

[dsd]

Please include that in the next gtk3 RPM. Then we can retest this.

13.2.0 build 1 includes the latest X server which has the final versions of Carlos's X fixes which we had not shipped until now.

So when build 2 happens with the GTK+ fix, we should be in good shape to re-evaluate this situation.

[erikos]

The patch that landed upstream is actually a merge of two patches we already had in our custom rpm: ​http://bugs.sugarlabs.org/attachment/ticket/4068/0001-gdk-protect-better-against-crossing-events-with-sour.patch and avoid_double_crossing_events_on_touch_end.patch

I removed these two and added the new one. rpm is building and will be in my public repo. gtk3-3.6.4-1.fc18.olpc2

[dnarvaez]

Sounds fixed.

[dnarvaez]

Milestone 0.98 deleted