[Pitch] Metatype Keypaths

Hello everyone,

The following is a pitch for metatype keypaths in the Swift language. We welcome any feedback or questions about this pitch!


[Pitch] Metatype Keypaths

Introduction

Key path expressions access properties dynamically. They are declared with a concrete root type and one or more key path components that define a path to a resulting value via the type’s properties, subscripts, optional-chaining expressions, forced unwrapped expressions, or self. This proposal expands key path expression access to include static properties of a type, i.e., metatype keypaths.

Motivation

Metatype keypaths were briefly explored in the pitch for SE-0254 and the proposal later recommended them as a future direction. Allowing key path expressions to directly refer to static properties has also been discussed on the Swift Forums for database lookups when used in conjunction with @dynamicMemberLookup and as a way to avoid verbose hacks like referring to a static property through another computed property. Supporting metatype keypaths in the Swift language will address these challenges and improve language semantics.

Proposed solution

We propose to allow keypath expressions to define a reference to static properties. The following usage, which currently generates a compiler error, will be allowed as valid Swift code.

struct Bee {
  static let name = "honeybee"
}

let kp = \Bee.Type.name

Detailed design

Metatype syntax

Keypath expressions where the first component refers to a static property will include .Type on their root types stated in the key path contextual type or in the key path literal. For example:

struct Bee {
  static let name = "honeybee"
}

let kpWithContextualType: KeyPath<Bee.Type, String> = \.name // key path contextual root type of Bee.Type
let kpWithLiteral = \Bee.Type.name // key path literal \Bee.Type

Attempting to write a metatype keypath without including .Type will trigger a diagnostic with a fix-it that recommends adding .Type:

let kpWithLiteral = \Bee.name // cannot refer to static member `name` on instance `Bee` without `.Type`.

Keypath expressions where the component referencing a static property is not the first component do not require .Type:

struct Species {
  static let isNative = true
}

struct Wasp {
  var species: Species.Type {Species.self}
}

let kpSecondComponentIsStatic = \Wasp.species.isNative

Access semantics

Immutable static properties will form the read-only keypaths just like immutable instance properties.

struct Tip {
  static let isIncluded = True
  let isVoluntary = False
}

let kpStaticImmutable: KeyPath<Tip.Type, Bool> = \.isIncluded 
let kpInstanceImmutable: KeyPath<Tip, Bool> = \.isVoluntary 

However, unlike instance members, keypaths to mutable static properties will conform to ReferenceWritableKeyPath because metatypes are reference types.

struct Tip {
  static var total = 0
  var flatRate = 20
}

let kpStaticMutable: ReferenceWriteableKeyPath<Tip.Type, Int> = \.total 
let kpInstanceMutable: WriteableKeyPath<Tip, Int> = \.flatRate 

Effect on source compatibility

This feature breaks source compatibility for key path expressions that reference static properties after subscript overloads. For example, the compiler cannot differentiate between subscript keypath components by return type in the following:

struct S {
  static var count: Int { 42 }
}

struct Test {
  subscript(x: Int) -> String { "" }
  subscript(y: Int) -> S.Type { S.self }
}

let kpViaSubscript = \Test.[42] // fails to typecheck

This keypath does not specify a contextual type, without which the key path value type is unknown. To form a keypath to the metatype subscript and return an Int, we can specify a contextual type with a value type of S.Type and chain the metatype keypath:

let kpViaSubscript: KeyPath<Test, S.Type> = \Test.[42]
let kpAppended = kpViaSubscript.appending(path: \.count)

Implications on adoption

Metatype keypaths are not back-deployable.

Implementation of metatype keypaths requires changes to the KeyPath type in the Swift standard library and a new runtime is necessary to take full advantage of this feature. Older compilers will continue to be supported but the correctness of operator comparisons (eg. Equatable or Hashable) will not be guaranteed for keypath references to static properties.

For example, Bank is a library compiled with Swift 5.10 or older:

public struct Bank {
  public static var checking: Double = 10.00
  public static var savings: Double = 200.00
}

Lets use Bank’s static properties in libraries compiled with the latest compiler that includes the metatype keypath implementation:

//--- UIManager.swift
import Bank
public let checkingUI = \Bank.Type.checking
public let savingsUI = \Bank.Type.savings

//--- StorageManager.swift
import Bank
public let checkingStorage = \Bank.Type.checking
public let savingsStorage = \Bank.Type.savings

When the above keypaths are compared elsewhere in the codebase, they may not reliably produce correct results unless the Bank library is updated to support the metatype keypath feature:

print(checkingUI == checkingStorage) // prints false, should print true
print(checkingUI != savingsStorage) // prints false, should print true

To leverage metatype keypaths effectively, library and framework binaries will need to be recompiled with a compiler that includes this implementation.

Future directions

Key Paths to Enum cases

Adding language support for read-only key paths to enum cases has been widely discussed on the Swift Forums but has been left out of this proposal as this merits a separate discussion around syntax design and implementation concerns.

Since references to enum cases must be metatypes, extending keypath expressions to include references to metatypes will hopefully bring the Swift language closer to adopting keypaths to enum cases in a future pitch.

Acknowledgments

Thank you to Joe Groff for providing pivotal feedback on this pitch and its possible implementation and to Becca Royal-Gordon for an insightful discussion around the anticipated hurdles in implementing this feature.

52 Likes

If this works, why doesn’t it apply to the full key-path in one expression? That is, there already must be a rule preferring instance key path elements over static ones; isn’t that sufficient here?

Are these comments correct? I’d want the two “checking” ones to compare equal and the two “savings” ones to compare unequal, and I’d expect a naive, backwards-compatible implementation to treat them both as unequal for now.


QoI suggestion

If I forget to write .Type, but the key path is otherwise unambiguous, can I get a nice fix-it?

5 Likes

Yes! I have added a diagnostic that should appear when .Type is forgotten to the pitch.

The operator results were a mistype. Both will incorrectly print false if a library is using an older compiler. I have updated that as well.

If this works, why doesn’t it apply to the full key-path in one expression? That is, there already must be a rule preferring instance key path elements over static ones; isn’t that sufficient here?

There is no rule that prefers instance members over static members for forming keypaths to subscripts, which is why we have this issue. I've updated this section to fix an error as well. Essentially, this will not typecheck:

let kpViaSubscript = \Test.[42]

as the solver is unable to pick between the two subscript options because we don't know the expected value type of this key path expression. So it does not pick the subscript with the String return overload. We could write:

let kpViaStringSubscript: KeyPath<Test, String> = \Test.[42]

to help the typechecker pick the String subscript because the key path value type is now String. If we need the Int value instead, we could specify a contextual type that includes the metatype as the value type:

let kpViaMetatypeSubscript: KeyPath<Test, S.Type> = \Test.[42]

then chain the Int returning component:

let kpAppended = kpViaMetatypeSubscript.appending(path: \.count)

or in one go:

let kpViaMetatypeSubscript: KeyPath<Test, Int> = \Test.[42].count

3 Likes

Thank you for the pitch!

I'm not sure I understand the "Implications on adoption" section correctly. It looks that metatype keypaths will have an observable behavior of Equatable that will depend on the operating system?

Also, one of the linked use cases mentions the interaction of metatype keypaths with SE-0252 Key Path Member Lookup. Does the proposal include support for the mentioned use case?

I think this is still wrong, the == one will incorrectly (/ conservatively) print false, but the != will correctly print false, just for the wrong reason. Or am I still misunderstanding?

This looks great, it's something that I've found myself needing many times over the years.

Also, strong +1 to exploring key paths to enum cases as a future direction.

4 Likes

Could you explain this further? I may be missing something obvious here, but comparing:

checkingUI != savingsStorage

will return true if all of the libraries are using the latest compiler, but may return false if the library where the static properties being referred is using a compiler without this implementation.

Edit: Are we referring to how Equatable flips the result for == and returns it for != ?

Yes, or if a library has not been updated and was compiled with a compiler that doesn't include this implementation but is used in a codebase that is regularly updated/recompiled.

Yes. This feature will allow us to refer to keypaths to static members from types with the @dynamicMemberLook attribute, like:

struct Weekday {
    static let day = "Monday"
}

@dynamicMemberLookup
struct StaticExample<T> {
    subscript<U>(dynamicMember keyPath: KeyPath<T.Type, U>) -> U {
        return T.self[keyPath: keyPath]
    }
}

let _ = StaticExample<Weekday>().day

Is there any precedent of previous compiler features that were shipped in this odd way in the past? If so, experience about the problems people have met would be interesting to hear. If not... Doesn't this sound concerning to anyone?

Does the proposal include support for [...] SE-0252?

Yes

That's good to hear :slightly_smiling_face:

1 Like

These are two different properties, referred to from two different libraries. On an OS with perfect information, they should be unequal because they are distinct. On an OS that has to treat them as opaque functions (like paths through computed properties), they should also be unequal because they might be distinct. So, shouldn’t we get the same answer either way?

I forget about this feature gap and then run into it every few months. This would be a very welcome improvement!

3 Likes

+1 very exciting, I’ve been wanting this for a long time to build a static level version of this: Combined: A type composed of other types (potential alternative to my Partial<T> type) · GitHub

There are all sorts of strange limitations in back deployment, but they aren't normally considered to be a matter for Evolution.

A little, but the alternative is to entirely limit use of metatype key paths to sufficiently new library compiler versions (which also means sufficiently new OS minimum deployment targets for key paths to OS types) solely to make sure the Equatable conformance handles key paths created in different places correctly. That seems like a pretty high price to pay for niche functionality—how often do you use KeyPaths as dictionary keys/set elements vs. all of their other uses (and how often do you rely on this working even for key paths generated in different modules)?

4 Likes

Thank you for the rationale. Indeed the use cases I envision for metatype keypaths don't use their equitability. I hope the people who do read this forum thread.

i can think of many use cases for Equatable on key paths, but they are all poorly-thought out attempts to wrap key paths in CodingKey(s). this is usually motivated by a desire to avoid SwiftSyntax macros.

please, let us make SwiftSyntax macros usable, instead of trying to match on key paths.

fwiw, I recently began a major effort on a large swiftui app to use keypaths as dictionary keys as the basis for a big improvement in styling. not sure that statics matter that much to me but i'm pretty enamored of the possibilities we're bringing to light with the technique.

The first thought after reading the title was "Finally enum support?" but ... no, unfortunately. :smiling_face_with_tear:

I'd like to share two updates to this pitch that we discovered after working through the implementation:

  1. Metatype keypaths are back deployable. Modules/libraries using older compilers will not need to re-compile to see correct operator overloading results. (The entire Implications on adoption section is no longer an issue).

  2. Diagnostics will show an error message if .Type is left out of a metatype keypath reference (where the first component is a static property), but a fix-it is not possible at this time. We'll see the following:

let kpWithLiteral = \Bee.name // error: static member 'name' cannot be used on instance of type 'Bee'

I will be leaving the pitch here as is but these discoveries will be integrated into the proposal draft.

13 Likes

I think it is strange to introduce the idea of types in the standard library that implement fundamental protocols like Equatable and Hashable that don't work correctly without providing some kind of a warning.

It seems completely reasonable to have sets of key paths or dictionaries keyed by key paths. The fact that KeyPath is Hashable implies it is intended to be used as a Hashable type.

As a developer, seeing KeyPath implements Equatable and Hashable, how would I have any idea that it may or may not really be properly equatable and hashable, depending on which type is at the root of the key path and how that type was compiled?

I understand the rationale for trying to avoid entirely limiting the use of metatype key paths for types in libraries compiled with older compiler versions.

Is there any way that diagnostics could be generated for uses of Equatable and Hashable that are going to be incorrect?

2 Likes
  1. Metatype keypaths are back deployable. Modules/libraries using older compilers will not need to re-compile to see correct operator overloading results. (The entire Implications on adoption section is no longer an issue).

I think my update here was not clear. The original pitch assumed that we would not be able to guarantee correct results for Equatable/Hashable comparisons for static properties declared in modules/libraries that were compiled with a compiler that did not include this feature's implementation.

Since implementing this feature, we have discovered that this is not true. Metatype keypaths will return correct results for Equatable/Hashable even if the static property is declared in a module compiled with an older compiler.

Is there any way that diagnostics could be generated for uses of Equatable and Hashable that are going to be incorrect?

So these will not be necessary. As a developer, you will be able to use metatype keypaths with the assurance that they will return correct results for operator comparisons just like any other keypath.

7 Likes