diff options
| author | Luka Zakrajšek <luka@bancek.net> | 2025-01-04 16:11:20 +0100 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2025-01-04 16:11:20 +0100 |
| commit | 57f41da13b10d909b85b7c335050e14fdb5b0d9b (patch) | |
| tree | 4b3d43b367d9bc1ba994c86f9bfe41d279bfa550 | |
| parent | cbaa0ad46f0eda2f3d9bcf8a42d6271e6760e578 (diff) | |
| download | candle-57f41da13b10d909b85b7c335050e14fdb5b0d9b.tar.gz candle-57f41da13b10d909b85b7c335050e14fdb5b0d9b.tar.bz2 candle-57f41da13b10d909b85b7c335050e14fdb5b0d9b.zip | |
Fix mistral attention on Metal (#2699)
Co-authored-by: Luka Zakrajsek <luka.zakrajsek@soniox.com>
| -rw-r--r-- | candle-transformers/src/models/mistral.rs | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/candle-transformers/src/models/mistral.rs b/candle-transformers/src/models/mistral.rs index f927f88b..8df73d61 100644 --- a/candle-transformers/src/models/mistral.rs +++ b/candle-transformers/src/models/mistral.rs @@ -262,7 +262,8 @@ impl Attention { .contiguous()?; let value_states = value_states .reshape((b_sz, q_len, self.num_kv_heads, self.head_dim))? - .transpose(1, 2)?; + .transpose(1, 2)? + .contiguous()?; let (query_states, key_states) = self.rotary_emb |