SE-0431: `@isolated(any)` Function Types

Evolution Proposal Reviews
1

Hello Swift community,

The review of SE-0431 "@isolated(any) Function Types" begins now and runs through April 9th, 2024.

Reviews are an important part of the Swift evolution process. All review feedback should be either on this forum thread or, if you would like to keep your feedback private, directly to me as the review manager via the forum messaging feature or email. When contacting the review manager directly, please put "SE-431" in the subject line.

Try it out

This feature is implemented on the main branch of the Swift compiler behind the experimental flag IsolatedAny, which can be enabled with the command-line argument -enable-experimental-feature IsolatedAny.

Toolchains that support this experimental flag are available at Swift.org - Download Swift.

What goes into a review?

The goal of the review process is to improve the proposal under review through constructive criticism and, eventually, determine the direction of Swift. When writing your review, here are some questions you might want to answer in your review:

What is your evaluation of the proposal?
Is the problem being addressed significant enough to warrant a change to Swift?
Does this proposal fit well with the feel and direction of Swift?
If you have used other languages or libraries with a similar feature, how do you feel that this proposal compares to those?
How much effort did you put into your review? A glance, a quick reading, or an in-depth study?

More information about the Swift evolution process is available at

swift-evolution/process.md at main · apple/swift-evolution · GitHub

Thank you,
Doug Gregor
Review Manager

10 Likes
3

Great step in further concurrency improvements!

Just one moment with syntax.
@isolated(any) is proposed for isolation erasure while keeping a space for @isolated(to:) as a future direction.
Currently functions / closures can be isolated to actor via @ActorName syntax, e.g. @MainActor. So we have different syntax for defining isolation.
Is there any thoughts or plans about something like @isolated(MainActor) as a more explicit, clear and unified syntax?

3 Likes
4

We could do that, but I don't think people in practice have any trouble understanding @MainActor as an isolation attribute. We try to avoid creating this kind of artificial style conflict.

There is an open question about how to spell dependent global actor isolation, though. We'd like to be able to provide something like this as a generic API (currently it's provided on MainActor as a special case):

extension GlobalActor {
  static func assumeIsolated(operation: @Self () -> T) -> T
}

It's not clear that allowing a type parameter like @Self to be used as an attribute name is a good idea; if we wanted to avoid that, we'd need to introduce something like @isolated(globalActor: Self).

4 Likes
5

According to my own experience and feedback from our team members who have started to learn new concurrency, overall concurrency syntax is felt less clear and consistent in comparison to other Swift parts. In general it is felt as a balance shift between clarity-brevity.

If @MainActor is not too difficult to remember as it is similar to DispatchQueue.main and Thread.main, the common rule can be ambiguous:

// 1.
func foo(operation: @SomeGlobalActor () -> Void) { ... } // less clear than @MainActor but `Actor` suffix helps

// 2.
@globalActor public final actor FavouritesManager: GlobalActor {} // declared somewhere in the project

func foo(operation: @FavouritesManager () -> Void) { ... } // even less clear

// 3. So this seems to be more clear:
func foo(operation: @isolated(FavouritesManager) () -> Void) { ... } // isolation is seen explicitly in source code

I clearly understand there is a huge expertise gap between those who learn new concurrency and those who design and implement it. Some of the theses and questions may seem naive.
My goal is to provide a feedback as a one from those who learn it and either suggest possible improvements from my point view or get some explanations from implementors to understand better why things are done as is.
Many thanks to overall Swift team for what you are doing and valuable conversations on this forum.

9 Likes
6

Just a quick note to say: many proposals are easy to understand and as such are a delight to read. This one is exemplarily. Thank you for that effort.

1 Like
7

While I don't think it's worth deprecating the { @MainActor in } It would be nice to be able to -- in future directions -- utilize the combination of those two things:

  • We're heading into the direction that isolation to MainActor.shared is the same as isolation to @MainActor,
    • "if the current context is isolated to a global actor T, the argument is T.shared" from SE-0420
    • with the assumption that shared really is meant to be "that one" and we're going to be relying on that.
  • the future direction in this proposal to allow @isolated(to: param), if/when we'd get there...

This would converge nicely with allowing:

func foo(
  operation: @isolated(to: MainActor.shared) () -> Void) { ... }
)

So we would not have to special-case the ability to pass the global actor type name, but just rely on the shared value every global actor has.

The assumeIsolated could then be spelled as

extension GlobalActor {
  static func assumeIsolated(operation: @isolated(to: Self.shared) () -> T) -> T
}

without new syntax.

Although I do realize the @isolated(to:) is quite some work to make happen, it seems like it would help converge a few styles of spelling similar things and help bridge the differences between global and normal actors :thinking:

Though that's relying on the future directions bit of @isolated(to:) here, and doesn't really stand in the way of the current @isolated(any) proposal IMHO.

edit: added SE-0420 reference

8 Likes
8

Currently global actors are defined via @globalActor and GlobalActor protocol, so compiler is aware of global actor Type and protocol's shared property allowing to define global actor isolation in a way like @SomeGlobalActor () -> Void.
Thus the compiler has already this abilities, seems like .shared can be omitted for global actors:

// 1.
func foo(operation: @isolated(MainActor) () -> Void) { ... } // compiler is aware of `.shared` property and known to be global 
func foo(operation: @isolated(MainActor.shared) () -> Void) { ... } // also allowed but `.shared` can be omitted
func foo(operation: @isolated(SomeGlobalActor) () -> Void) { ... } // compiler is aware of `.shared` property and known to be global 

// 2.
extension FavouritesManager {
  static let products = Self.init(...)
  static let receipts = Self.init(...)
}
func foo(operation: @isolated(FavouritesManager.products) () -> Void) { ... } // isolated by first instance stored in `static let` property 
func foo(operation: @isolated(FavouritesManager.receipts) () -> Void) { ... } // isolated by second instance stored in `static let` property

// 3.
func foo(operation: @isolated(any) () -> Void) { ... } // type erased
9

I'd argue against this because it yet again is "special" spelling IMHO.

Generally in Swift you can not just spell "TheType" and you'd normally spell TheType.self and I, personally at least, really don't enjoy when attributes we introduce have "syntax that does not exist elsewhere". Attributes sadly often fall into this -- (I'm looking at you @available with it's very bespoke syntax).

Either way, as this is a future direction in this proposal I'm not sure we should be focusing on this piece and focus on the @isolated(any)[1] being proposed perhaps?

[1](Which I guess, funnily enough also is a "special spelling" heh -- but not sure introducing more noise here specifically would help it much (e.g. @isolated(to: .any) I don't think helps much, unless we'd want to change how we build annotations entirely and always try to have them be "normal" enum values and parameters...)

2 Likes
10

I thought it would be legal because current @MainActor doesn't specify .self or .shared.
Now I get it, thanks for claryfying this:

Shure. I'm totally happy with the proposal and @isolated(any), just wanted to discuss one moment from future direction. You've provided enough explanations :pray:.

1 Like
11

If only G specifies @isolated(any), then the behavior depends on the specified isolation of F:

  • If F has an isolated parameter, the conversion is invalid.

What's the rationale behind this limitation? My intuition is that isolation to a parameter is just another form of isolation to a specific actor, which this proposal already supports for captures. Is there some consequence that I'm not thinking of?

12

Well, consider an example. Suppose I have a @Sendable (isolated MyActor) -> (), and I convert that to @isolated(any) (MyActor) -> (). What is the isolation of that function? It's not isolated to some specific actor — it's isolated to whatever actor you pass in as its argument. So what does isolation return?

Now, I can imagine a feature that's like "here's an algorithm for dynamically deciding the isolation of this function given all its arguments". I think that's a different and more complicated feature, though. Moreover, in practice I think it probably wouldn't be very useful, because I don't think it's common to have a function that takes an actor argument like this but not know whether the function will be isolated to it; and if you're "partially applying" the function to a specific actor, you can express that with an isolated capture:

let fn: @Sendable (isolated MyActor) -> () = ...
let myActor = ...

let convertedFn: @isolated(any) () -> () =
  { [isolated myActor] in fn(myActor) }
2 Likes
13

Thank you, that clears it up! I think I glossed over the fact that the specific actor is chosen by the caller instead of the callee in this case. Maybe this could be quickly mentioned in the proposal, but it's not that important.

14

Overall I think the proposal & design looks great, but I'd like to echo a similar sentiment as Dmitriy.

Most of Swift has the property of gracefully introducing the user to new concepts when they're needed, often in fixits or error messages, and/or by making common enough harder concepts (like concurrency) syntactically simple. I'm not getting the sense however that it will be obvious to the average dev team member, even one working on a library, to utilize @isolated(any) in the cases where it should be included here unless they have read this proposal (or future documentation), so this seems like this could be a common performance miss. Will there be mechanisms (or tooling) for communicating this more directly or will this need to just be one of those culturally encoded "best practices" in Swift?

15

Great proposal that adds a missing feature!

I'm wondering though how we could extend this to other types than just functions.
Swift has recently ported features that were once only available to functions/closures to other places e.g.:

  • Typed throws have made rethrows possible in protocols and generics. Also replaces @rethrows for protocols.
  • ~Escapable is being discussed on the forums as well that brings the power of @nonescapble closures to protocols and generics.

On the other hand, protocols/structs/enums/classes have features that aren't available to closures. Closures can not properly participate in the generics system because a function signature is not a valid generic constraint. Protocols are sometimes needed to workaround this limitation.

This has come up recently in another thread and I have written down some of the use cases where functions/closures currently can't be used and we need to fallback to protocols: FunctionalProtocols for Swift - #15 by dnadoba

I can imagine that we will eventually need to bring this and other isolation features to other types in general, potential make it part of the generics system so this information can be propagate through chained operations.

This is likely out of scope for this proposal but I would love to see something in the Future directions section.

16

Allowing generics to talk explicitly about abstract isolations in the same as they reason about, say, typed errors would require value-dependent types, which are already discussed in Future Directions; the quick summary is that it's inherently conceptually difficult and not likely to be something we'd pursue.

Some kind of generic function constraint is something we could think about, but I don't think it's really something that needs to be developed in this proposal. It would have to consider isolation, but that's not new.

1 Like
17

i realize that the nominal review period for this is over, so i apologize if this feedback would be better placed elsewhere, but i had a few thoughts/questions on this and some related proposals.

in particular, i'm wondering how the proposal is intended to interact with the recently-approved 'task executor preference' API (SE-0417).

consider the following example:

let op: @Sendable @MainActor () async -> Void = {
    MainActor.assertIsolated()
    print("main actor")
}

Task(
    executorPreference: CustomTaskExecutor.shared,
    operation: op
)

based on my reading of the two documents, i would expect the formal isolation of the operation to be resolved to the main actor, and thus would expect the Task to be directly enqued there, effectively bypassing the explicitly-specified TaskExecutor preference. is this interpretation accurate? perhaps it is expected given the current implementation status of things, but this did not appear to be how some of the recent development toolchain snapshots behave.

an additional thought, which admittedly is a bit vague, is in regards to potential 'macro-scale' behavioral changes that this feature may induce in existing code. consider another (somewhat contrived) example where we enqueue a number of tasks of 2 different priorities to be run on the main actor:

func test_priorities() async {
    let iterationCount = 100
    var done: (() async -> Void)?
    for i in 1...iterationCount {
        let pri: TaskPriority = (i > iterationCount / 2) ? .high : .low
        
        // aside: an explicit type annotation is needed for some reason.
        // @MainActor isolation appears to be dropped without it.
        let op: @Sendable @MainActor () async -> Void = {
            let qos = qos_class_self()
            print("running task: \(i), pri: \(pri), qos: \(qos)")
        }

        let t = Task(
            priority: pri,
            operation: op
        )
        done = { await t.value }
    }

    await done?()
}

today, due to the initial 'hop' to the generic executor, this form of code will often effectively behave somewhat like a DispatchWorkloop. that is, the Tasks will (often) be run in decreasing priority order. with the enqueuing logic change proposed here, this will no longer be the case, and in this example, all the low priority Tasks will be executed before the high priority ones.

now, i think in this example this is a non-issue (the effective priority of the Tasks is escalated since they run on the main thread), but it at least raises the question of whether this change will potentially lead to priority inversions that did not used to exist, or more generally will have observable and undesirable changes on existing code.

overall this seems like a good addition to the language, and personally i think making initial execution order match Task initialization order where that is possible and reasonable would be a huge win, but do wonder about potential impacts to existing code.

1 Like
18

That’s right. Function isolation should always “win” over the task executor when they’re in conflict. They’re not always in conflict, though — actors will by default run on task executors. If you’re not seeing that, we’ll look into it.

19

Almost any scheduling change can theoretically have undesirable changes on existing code. It's not something that's worth keeping oneself up all night over.

20

I'd like to open up the conversation about the spelling here.

The spelling in the current proposal, @isolated(any), intentionally carves out @isolated as a namespace for similar attributes for controlling isolation. For example, we could use @isolated(caller) as a shorthand for SE-0420's param: isolated (any Actor)? = #isolation, or @isolated(to: value) in the future direction of value-dependent function isolation. However, we have existing uses of isolated and nonisolated that don't use an @, and while it's not grammatically problematic to distinguish features by @-vs.-no-@, it is likely to be confusing to people using the language. (This is why I include nonisolated — formally, yes, it's a totally different keyword, but people would be surprised for it to have different rules from isolated.) We probably don't want to create this artificial question of whether isolated starts with an @.

At the same time, I'm very reluctant to just remove the @. Originally, I added the @ in order to solve a thorny parsing problem; I think that problem can probably be overcome, and I don't think we should worry too much about it. However, I think it's wrong in a much deeper way for isolated and isolated(any) to both exist but apply in very different ways in the language:

// `isolated` is a parameter modifier.  It can only be used in this
// way on a parameter and isn't reflected in the type of a reference
// to `param`.
func foo(param: isolated MyActor)

// `isolated(any)` is part of the type.  It can be used anywhere you
// can write a type, and of course it is reflected in the type of a
// reference to `param`.
func bar(param: isolated(any) () -> ())

We should not muddy the waters between type attributes and parameter modifiers this way.

If we want to avoid both of these problems, I think the best approach is to use a different spelling entirely. We can still have isolated in the name in order to make the attribute more self-documenting, but we should try to avoid making it look like the existing modifier. That would suggest something like @dynamicallyIsolated. If we introduce other isolation-controlling attributes in the future, we can follow that same pattern with e.g. @callerIsolated.

7 Likes