Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Added HasCallStack to partial functions #493

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
dwincort wants to merge 5 commits into haskell:master
base: master
Choose a base branch
Loading
from dwincort:master

Conversation

Copy link

@dwincort dwincort commented Jan 13, 2018

As discussed on Issue #489, this PR adds HasCallStack to the constraints of partial functions. I think I got all of the relevant partial functions, but I'd be happy for someone to double check that I didn't miss anything.

I ran the benchmarks (stack bench) before and after these changes and have attached them (before_hascallstack.bench.txt, after_hascallstack.bench.txt). I scanned through the results, trying to pay attention to tests on partial functions specifically, but I didn't notice any changes (outside of expected deviations within the margin of error).

Please let me know if there's more that I can do to help get this PR successfully merged.

Copy link
Contributor

treeowl commented Jan 13, 2018

You're going to need to wrap all those pieces in CPP restricting in to appropriate __GLASGOW_HASKELL__ versions.

Copy link
Author

Is it a __GLASGOW_HASKELL__ version or is it MIN_VERSION_base(4,9,0)?

Thanks!

Copy link
Contributor

treeowl commented Jan 13, 2018 via email

Glasgow Haskell, because it really relies on GHC-specific features.
...
On Jan 13, 2018 1:33 PM, "Daniel Winograd-Cort" ***@***.***> wrote: Is it a __GLASGOW_HASKELL__ version or is it MIN_VERSION_base(4,9,0)? Thanks! — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#493 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABzi_UXmc9vmlNSnH23l5tJJgPHiyzIZks5tKPcXgaJpZM4RdSDv> .

Copy link
Contributor

@treeowl treeowl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The questions about update and deletion apply across the board.

updateAt :: (k -> a -> Maybe a) -> Int -> Map k a -> Map k a
#endif
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better to just make this function total by making an update at a bad index do nothing?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is definitely a more interesting question than just where to add HasCallStack and might make more sense for a different issue/PR. Personally, I'm not sure I've ever used the ***At functions, so I'm not really the target audience to be weighing in on this decision. Is there a way you could get community feedback about such a change? Maybe Haskell-cafe?

@@ -1588,7 +1607,11 @@ updateAt f !i t =
-- > deleteAt 2 (fromList [(5,"a"), (3,"b")]) Error: index out of range
-- > deleteAt (-1) (fromList [(5,"a"), (3,"b")]) Error: index out of range

#if __GLASGOW_HASKELL__ >= 800
deleteAt :: HasCallStack => Int -> Map k a -> Map k a
#else
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same question here.

@@ -2661,7 +2692,11 @@ mergeA
-- @only2@ are 'id' and @'const' 'empty'@, but for example @'map' f@,
-- @'filterWithKey' f@, or @'mapMaybeWithKey' f@ could be used for any @f@.

#if __GLASGOW_HASKELL__ >= 800
mergeWithKey :: (HasCallStack, Ord k)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ehhhhh... Not sure we should bother with this one. mergeWithKey is for users who don't mind the risk of being shot in the foot.

@@ -4300,7 +4348,11 @@ zipWith f s1 s2 = zipWith' f s1' s2'
s2' = take minLen s2

-- | A version of zipWith that assumes the sequences have the same length.
#if __GLASGOW_HASKELL__ >= 800
zipWith' :: HasCallStack => (a -> b -> c) -> Seq a -> Seq b -> Seq c
#else
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No no no. This is an internal function and is only used when we know it will succeed.

@@ -4498,7 +4554,11 @@ draw (PQueue x ts0) = x : drawSubTrees ts0
-- | 'popMin', given an ordering function, constructs a stateful action
-- which pops the smallest elements from a queue. This action will fail
-- on empty queues.
#if __GLASGOW_HASKELL__ >= 800
popMin :: HasCallStack => (e -> e -> Ordering) -> State (PQueue e) e
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No no no. This is an internal function.

Copy link
Contributor

treeowl commented Jan 15, 2018 via email

You're right; separate PR. I guess I can ask the libraries list.
...
On Jan 15, 2018 12:42 PM, "Daniel Winograd-Cort" ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In Data/Map/Internal.hs <#493 (comment)>: > updateAt :: (k -> a -> Maybe a) -> Int -> Map k a -> Map k a +#endif This is definitely a more interesting question than just where to add HasCallStack and might make more sense for a different issue/PR. Personally, I'm not sure I've ever used the ***At functions, so I'm not really the target audience to be weighing in on this decision. Is there a way you could get community feedback about such a change? Maybe Haskell-cafe? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#493 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABzi_ZY7CVkokVE5QHC8ZpqGPor5xnfNks5tK44ggaJpZM4RdSDv> .

findMin :: IntSet -> Key
#endif
findMin Nil = error "findMin: empty set has no minimal element"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the call stack grow through the recursion here or elsewhere? If so, that's a big problem. You can check the Core to be sure. We know there are no Nils except at the root, but GHC does not! If the stacks build, you'll need to restructure the functions to fix that. Watch out for performance. If the times for the current benchmarks exercising these functions are too short to trust, consider adding more benchmarks.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wow, yes it does. I didn't realize it would do that. The fix is to use an internal go function. I'll update the PR with that in a moment.

On a related note, I realized that my HasCallStack for Map.! was wrong because Map.! calls find which actually throws the error. Is there a reason it uses find rather than lookup? My inclination is to remove find altogether (it's only used by ! and looks identical other than the Maybe wrapper) and replace with lookup---then ! can call error itself, which should then make HasCallStack work the way we want.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I just noticed the [Note: Local 'go' functions and capturing] -- I'll read that and try to make sure I'm not doing anything stupid.

@@ -4457,7 +4505,11 @@ unstableSortBy cmp (Seq xs) =
-- | fromList2, given a list and its length, constructs a completely
-- balanced Seq whose elements are that list using the replicateA
-- generalization.
#if __GLASGOW_HASKELL__ >= 800
fromList2 :: HasCallStack => Int -> [a] -> Seq a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an internal function and shouldn't get a call stack. It's used to implement the IsList instance. Arguably, the method it's used to implement should get a call stack, but that method isn't really intended to be called directly in user code anyway, and GHC will always call it correctly.

Copy link
Contributor

treeowl commented Jan 15, 2018 via email

You may be able to avoid capturing issues by giving the `go` functions full signatures, making it clear they don't take call stacks. Check Core often, maybe look at STG occasionally, and benchmark.
...
On Jan 15, 2018 2:11 PM, "Daniel Winograd-Cort" ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In Data/IntSet/Internal.hs <#493 (comment)>: > findMin :: IntSet -> Key +#endif findMin Nil = error "findMin: empty set has no minimal element" Oh, I just noticed the [Note: Local 'go' functions and capturing] -- I'll read that and try to make sure I'm not doing anything stupid. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#493 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABzi_ejufDBgEXQbVqoa6kUyuslx4sKpks5tK6LRgaJpZM4RdSDv> .

(!) m k = find k m
(!) m k
| Just a <- lookup k m = a
| otherwise = error ("IntMap.!: key " ++ show k ++ " is not an element of the map")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using lookup will incur a Just allocation. One possible solution (for very recent GHC versions only) would be to define a lookup# function producing an unboxed sum:

type Maybe# a = (# Void# | a #)
pattern Nothing# :: Maybe# a
pattern Nothing# <- (# _ | #) where
 Nothing# = (# void# | #)
pattern Just# :: a -> Maybe# a
pattern Just# a = (# | a #)
lookup# :: Key -> IntMap a -> Maybe# a
lookup# k m = ...
lookup k m = case lookup# k m of
 Nothing# -> Nothing
 Just# a -> Just a

The (potentially) great thing about this is that the Just gets applied on the "outside", so inlining and the case-of-case transformation will end up making it go away altogether:

m ! k = case
 (case lookup# k m of
 Nothing# -> Nothing
 Just# a -> Just a) of
 Nothing -> error ...
 Just v -> v
-- ==> case-of-case, case of known constructor
m ! k =
 case lookup# k m of
 Nothing# -> error ...
 Just# a -> a

Unlike a CPS version, this theoretically helps user-written code as well. However, I have not actually tried benchmarking anything like this. And the CPP is potentially troublesome too.

Copy link
Author

@dwincort dwincort Feb 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems cool -- it seems like it could be a fairly big change that would even prevent an allocation when one uses lookup (or related functions that return Maybe values) and immediately unwraps -- but it's a bit big for what I was hoping to contribute just now (also, a bit over my head). I think I'll take the easy way out and go back to using a find function to get a result without the extra allocation. Perhaps my next PR will be to overhaul the backend to use unboxed sums where possible :).

@@ -887,9 +887,20 @@ atKeyIdentity k f t = Identity $ atKeyPlain Strict k (coerce f) t

#if __GLASGOW_HASKELL__ >= 800
updateAt :: HasCallStack => (k -> a -> Maybe a) -> Int -> Map k a -> Map k a
updateAt = go where
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this actually give the right call stacks? Thinking about this more carefully, I suspect you want to check if the operation will succeed, and only then call a go function that takes no call stack. The alternative would be to call the go function using withFrozenCallStack, which I suspect will be more expensive. Benchmarks may tell.

Copy link
Author

@dwincort dwincort Feb 6, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My tests show that, by default, functions like go do not have the HasCallStack constraint unless it is explicit in their type signatures. Therefore, this seems to give the right call stack. There's no need to check if the operation succeeds or not first, and because go doesn't have the constraint, there's no need for withFrozenCallStack.

Copy link
Contributor

treeowl commented Jan 22, 2018

Are you making any progress? I'd love to get this into the next release, but that's probably going to happen in the morning!

Copy link
Author

I'm sorry, but I got caught up in other work this past week and haven't worked on this at all. I guess it will have to wait until the next release :/.

Copy link
Contributor

treeowl commented Jan 22, 2018 via email

That's fine. Just do the best you can.
...
On Jan 22, 2018 12:03 PM, "Daniel Winograd-Cort" ***@***.***> wrote: I'm sorry, but I got caught up in other work this past week and haven't worked on this at all. I guess it will have to wait until the next release :/. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#493 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABzi_c75_mZ3BnpLEp2e55_OwTuJm3S4ks5tNL9ugaJpZM4RdSDv> .

Copy link
Author

dwincort commented Feb 6, 2018

More generally, I must confess that I really don't know how to check Core. I think I know how to look at Core (-ddump-simpl?), but I don't know what to do with it or how to tell if it's good or bad.

Copy link
Contributor

treeowl commented Feb 8, 2018

-ddump-simpl indeed. I suggest that for a while you should add -dsuppress-all -dno-suppress-type-signatures. You want to look for fishy things like recursively growing call stacks.

Separately, you should test each function with bad arguments of all relevant sorts and make sure the call stack indicates that the problem is with the bad code that called the partial function, not the partial function itself.

Copy link

bollu commented Dec 24, 2018

What is the status on this? I wound up writing custom wrappers around some of the functions in Data.Map for stack traces, but it would be awesome if this were upstreamed.

Copy link
Contributor

treeowl commented Dec 24, 2018 via email

No one has done the grunt work to check on whether this is optimized the way we want and whether the performance regressions are tolerable.
...
On Mon, Dec 24, 2018 at 3:52 PM Siddharth ***@***.***> wrote: What is the status on this? I wound up writing custom wrappers around some of the functions in Data.Map for stack traces, but it would be awesome if this were upstreamed. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#493 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABzi_WC-HcAgRNGMV6WTKUp-wxfYMOGJks5u8T55gaJpZM4RdSDv> .

Copy link

bollu commented Dec 24, 2018
edited
Loading

Is it necessary to pay a performance cost? Can we not expose a Data.Map.{Strict, Lazy}.Debug that contains the functions that expose debug info?

Copy link
Contributor

treeowl commented Dec 25, 2018 via email

Regardless, the way we proceed will depend on the performance cost. If it's low enough, I think it's okay to force users to accept it. Otherwise not. But either way, we want to be sure that the cost is as low as possible. In particular, we definitely want to make sure that we *never* build up call stacks recursively. The call stacks are for debugging code that *uses* the package, and explicitly *not* for debugging `containers` itself.
...
On Mon, Dec 24, 2018 at 5:13 PM Siddharth ***@***.***> wrote: Is it necessary to pay a performance cost? Can we not expose a Data.Map.{Strict, Lazy}.Debug that contains the functions that expose debug info?/ — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#493 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABzi_dz5M8A6kX9EeWay0qg1O9RFAXR0ks5u8VGHgaJpZM4RdSDv> .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@treeowl treeowl treeowl requested changes

Assignees
No one assigned
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add HasCallStack to partial functions

AltStyle によって変換されたページ (->オリジナル) /